ObjectiveFS

Memory Optimization Guide

ObjectiveFS uses memory to cache filesystem data and metadata locally to improve performance and to reduce the number of S3 operations.The amount of memory used by ObjectiveFS is based on the memory cache size, number of S3 objects used by your filesystem and the amount requested by the Linux kernel.

The memory cache size can be adjusted using the CACHESIZE environment variable (see details). You can find out the number of S3 objects used by your filesystem by looking at the IUsed column in df -i.

Recommendation

We recommend running on servers with at least 1GB+ of memory.To make sure you have enough memory for your other applications, you can use CACHESIZE + (500MB to 1GB depending on your filesystem size) as an estimate for each ObjectiveFS mount.The CACHESIZE used by each mount is logged upon filesystem mount (see log info).

Lowering Memory Usage

If you need to lower the memory usage of ObjectiveFS, we recommend using one or more of the following steps.

1. Use a smaller memory cache

Memory cache can be adjusted using the CACHESIZE environment variable
The minimum value for CACHESIZE is 64M
The CACHESIZE value is also printed upon filesystem mount, either on screen (when running in foreground) or in the log file (when running in background)
Reference: Memory Cache User Guide

2. Enable disk cache

Use the local instance store as disk cache
The disk cache maintains good performance even with a smaller memory cache
Reference: Disk Cache User Guide

3. Avoid mounting multiple filesystems on a small machine

Each ObjectiveFS mount has its own memory usage
When calculating memory usage, please take into account the total memory needed by all your filesystems

4. Enable compaction

Compaction is enabled by default and will reduce the number of S3 objects used by your filesystem over time
Reference: Compaction User Guide

5. For read-only servers

If your server only needs to read your filesystem, you can also mount your filesystem with the read-only option (mount -o ro). (see mount options)
```
$ sudo mount.objectivefs mount -o ro <filesystem> <dir>
```

6. Other options

Verify you are using the latest ObjectiveFS version to get the latest performance optimizations.
Use EC2 spot instances with similar price but larger memory and/or larger instance store.
Use previous generation instances with bigger memory for the same price.

If you have questions about how to optimize memory for your filesystem and use cases, please feel free to contact from here. We are happy to work with you to find a good solution for your use case.

References

by ObjectiveFS staff, February 2, 2016
ObjectiveFS is a shared file system for OS X and Linux that automatically scales and gives you scalable cloud storage.If you have questions or article idea suggestions, please email us at お問い合わせはこちら