How To Optimize Large S3 API Costs using Alluxio

This article describes how engineers at Datasapiens brought down S3 API costs by implementing Alluxio as a data orchestration layer between S3 and Presto. The main conclusion from these use cases is that using AlluxIO has the following benefits: lower latency in data processing pipelines. The infrastructure costs were above expectations due to S3 cost increases, making up a large proportion of the overall cost (described in detail below) This architecture achieves faster performance and reduced compilation times. In addition, we saw a drop in infrastructure costs due to the increase in S3 costs.