Case Study: Online File Storage on S3

Serverless apps are very popular right now. Developers can scale up advanced business logic quickly and lean on cloud providers like AWS to do the tedious work. With the advent of serverless, bespoke business have become commonplace, as businesses can easily develop and maintain these apps with limited developer resources. Let me share a recent serverless app I built with you and highlight its successes.

The App

The “File Manager” a tool I built for business which is essentially a variant of Google Drive or Dropbox. The premise is this: many businesses, like construction companies, need to take pictures and videos of their work to track their progress. People in office can create reports or auditing and forward this information to stakeholders. In my particular case, the app had to be highly interoperable with their office tooling (Quickbase in this case).

The File Manager is built entirely with AWS serverless architecture. It offers a web page as a single page application for users upload and view media. It has an authentication system to validate the user is a part of that business and has access to their ERP (Quickbase). In addition to uploading and retrieving files, via AWS S3, it performs optimization tasks on uploaded files. When a construction worker uploads a picture of his or her handiwork on a kitchen island, the file might be 10 megabytes. I made sure the File Manager would provide thumbnails so that when they are in low cell coverage, they can still access their photos quickly.2

Similarly, if an electrician needs to make a video of where the conduit leads, the File Manager has serverless automations in place to compress the video down to mobile connectivity standards. Oftentimes, recorded videos are hundreds of megabytes, which would mean waiting an hour for example, on a slower LTE connection. In the serverless world, I simply tacked on a script to utilize streaming algorithms and reduce the wait time almost entirely for my users.

Serverless Benefits

The File Manager has been so straightforward to build because it was serverless. All files were saved in S3 so I didn’t have to think about hosting a file server. Event based systems like SQS mean I can just write a script for some event (like video uploads for example). The frontend is served up on a CDN, so all that’s left to implement is how to run my business logic.

AWS Lambda does this well. It offers users a computing environment that is only run when a request is made. That means gone are the days where servers need down time for upgrades. No more scaling up servers to larger hardware–AWS lets lambda functions run concurrently. (for more info on what responsibilities AWS manages, check out their writeup)

Proof is in the Pudding

On the order of tens of dollars per month, I am seeing some very high activity. I have a Grafana dashboard set up to monitor the app. Here are some metrics I have been tracking:

Above you can see what my actual login count was every 12 hours. You can see that in the past couple weeks, daily logins have been around 400.

And here is a graph of the daily uploads. You can see that the uploads are hovering around 5000 a day. Lot’s of traffic!

My app stores a significant amount of data, the latest metric shows just over 2.5 terabytes of data stored.

I’ve recently started tracking the correlation between logins and file uploads. Sometimes my app needs to handle big spikes in file uploads.

How is it so cheap

The monthly spend for this app has not yet reached $100/month. Assuming we did spend $100/month, and we expect 400 active users, that’s $0.25 per user per month. Or put another way, paying $100/month for 2.5TB of storage is about $0.04 per gigabyte. That’s a pretty good deal.

When considering cloud costs for serverless apps, the majority of the cost will probably be data transfer, s3 costs, and lambda costs. AWS only charges for data transfer out (so, downloads in my case), and it’s $0.09 per gigabyte. Lambda costs are $0.00001667 per gigabyte-second after the first 400,000 gigabyte-seconds. S3 costs are $0.023 per gigabyte.

If you do some quick math you can see that hosting costs are dramatically cheaper than running servers, and because it’s serverless, developers don’t have to focus on maintenance. And less infrastructure logic means more time for business logic

How I built it

With several serverless apps under my belt, there are a few best practices I stick to in order to ensure a successful business app. First, starting with a clear, simple use case to build off of. This allows me to focus on the core business logic and not get bogged down in the details. Second, I always use a CI/CD pipeline to deploy my apps. This ensures that I can quickly iterate on my app and get it into the hands of my users. Finally, I always use a monitoring tool like Grafana to keep an eye on my app’s performance. While serverless apps mean less maintenance, they still need to be monitored to catch problems before they happen.

Conclusion

Serverless apps allow for tremendous flexibility because they are lightweight and cheap. Unless you are a tech company, you might not have the means to spin up a fully fledged set of servers to tackle business problems, but with the right toolset and some careful planning, serverless apps quickly yield impressive results.