Troubleshooting in a Large System
Our server has 75M daily TPS which generates a lot of ops work, I’m often asked “What do you guys do when you’re on call?” this blog will demon straight how we do troubleshooting in a distributed system.
Speed Up Your AWS S3 Client
- Java, AWS, S3, Performance
Our team recently had a performance issue with some data processing. Every day we have 24 files of 30GB generated in S3, and we’re having a Fargate cluster to download and process those data, it takes 12 hours to processing all 600+GB files, which is too slow as we want to increase the size of file for processing. After a serial of improvement, we successfully reduce the processing to 1.5 hours.
This is a sample project to explain what improvement we’ve done: https://github.com/ADU-21/s3-parallel-download
AWS DynamoDB Study Note
DynamoDB was announced by Amazon CTO Werner Vogels on in 2012, 14 years after NoSQL was proposed in 1998. It supports key-value and document-oriented structure storage.
AWS Step Function
Our client recently has a deployment system that has been in use for more than 10 years and wants to migrate to the cloud, this blog shows how we migrate it step by step from a huge single application to serverless by AWS Step funtion.
AWS Data Analytics
Big things, fast, minimal set up, maximum security, low cost.