Performance: Amazon EFS vs ObjectiveFS
When scaling web servers and content management systems (CMS), a shared filesystem is useful for storing and sharing file assets. Performance for small files is an important factor to consider when choosing a shared filesystem for web server and CMS workloads. We compared the performance of two shared filesystems that are commonly used to scale these workloads on AWS EC2 instances: Amazon Elastic File System (EFS) and ObjectiveFS. For details on how Amazon EFS differs from ObjectiveFS, see here.
Setup
We set up an r3.xlarge EC2 instance in the us-east-1 region. For EFS, we used a filesystem in the us-east-1 region with general purpose performance mode. The general purpose performance mode was selected because Amazon states it is better for latency-sensitive use cases like web servers and CMS than the max I/O performance mode. For ObjectiveFS, we used an S3 bucket in the us-east-1 region as the object store with the default memory cache setting and multithreading enabled. For the disk cache test, we used the local instance store on the EC2 instance.
Note: For EFS, we kept the test within the burst time limit, to get the best possible EFS performance. For a 100GiB filesystem, the EFS burst limit is 72 minutes per day. After that, Amazon EFS performance is limited to 5MiB/s. For more details, see the Throughput Scaling section of Amazon EFS Performance. Note that ObjectiveFS does not have any burst limits and can sustain high throughput consistently 24x7.
Setup Summary
Component | Settings |
---|---|
EC2 instance | r3.xlarge EC2 instance in us-east-1 |
Amazon EFS | general purpose performance mode in us-east-1 |
ObjectiveFS | multithreading, s3 bucket in us-east-1 (disk cache) |
Read Performance
We compared the read performance for random read access of small files, similar to a web server workload. The test performs 16 parallel reads to 10K small files on the filesystem with an 80/20 access pattern, and the latency and throughput are measured. Each filesystem contains the linux 4.7.2 kernel source tree with ~54K files in 3657 directories; the total size is 726MB and all files are smaller than 2MB.
The tables below show the latency and throughput results. For ObjectiveFS, it shows the results when the caches are not warmed (e.g. on initial boot), when the disk cache is warmed (e.g. after system reboots), or when the memory cache is warmed (during use).Kernel caches are dropped using echo 3 > /proc/sys/vm/drop_caches
before each run for both filesystems.
Latency – small random reads
Filesystem | Latency (95th-percentile) | Improvement |
---|---|---|
EFS | 81 ms | 1x |
ObjectiveFS (cold cache) | 80 ms | 1x |
ObjectiveFS (warm disk cache) | 6 ms | 13x |
ObjectiveFS (warm memory cache) | <1 ms | 81x |
Throughput – small random reads
Filesystem | Read Throughput | Improvement |
---|---|---|
EFS | 3 MB/s | 1x |
ObjectiveFS (cold cache) | 10 MB/s | 3x |
ObjectiveFS (warm disk cache) | 54 MB/s | 18x |
ObjectiveFS (warm memory cache) | 387 MB/s | 129x |
Write Performance
We also compared the write performance with small files. This test writes ~54K small files in the filesystem by unpacking the linux 4.7.2 kernel source tree. We measured the time it takes to untar these files, totaling 762MB on each filesystem and compared the throughput. For writes, ObjectiveFS performance is independent of the cache settings.
Performance – untar
Filesystem | Time | Write Throughput | Improvement |
---|---|---|---|
EFS | 2283 s | 0.3 MB/s | 1x |
ObjectiveFS | 33 s | 22 MB/s | 69x |
Conclusion
For web server workloads, ObjectiveFS has higher performance and lower latency compared to Amazon EFS. ObjectiveFS read latency is up to 81x better than Amazon EFS. The read and write throughputs are also significantly better (up to 129x for reads and 69x for writes).
To see how large file workloads compare, see Large File Performance: Amazon EFS vs ObjectiveFS.
To see how ObjectiveFS performs for your workload, you can download a copy and try it with our free trial.
by ObjectiveFS staff, September 20, 2016
ObjectiveFS is a shared file system for OS X and Linux that automatically scales and gives you scalable cloud storage. If you have questions or article idea suggestions, please email us at お問い合わせはこちら
Reference: Linux kernel source tree: https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.7.2.tar.xz