s3 and conversion server
Jamroom Developers
Hey Brian,
Just wanted to follow up on an earlier part of this thread regarding storage. I know you're still working on the cloud documentation, but I didn't see anything in there related to storage.
I'd like to be able to use S3 at this point. I was thinking to just create a module that will write write files from the /data directory to the s3 bucket. The questions are:
I know you have different plans giving Jamroom users more options for storage via XtremeFS. I've also been reading up on both s3fuse and s3fs. Also note that s3fuse is considered an alpha software. Neither has had any updates in a while. They both have a couple of bugs. And apparently you still have the same limitations as if you used the API. Not sure if its a good idea to use in production.
1.Can you recommend an easy way to hook into read/writes of the /data folder to using the s3 api?
2. Will using the s3 API cause problems for me scaling other parts of the cloud software?
3. If s3 fuse is easier/better/faster, can you make suggestions on how to best integrate it into JR so as to save my data there? I'm not clear on where the right hooks are; weather via api or fuse.
In either scenario, we need to consider that we don't want to write everything to s3, only data folder. Other files that have incremental changes are not suited for s3 because you have to read and write the whole file. This is the case even if you use mount since it is just a front end to s3.
I 'm getting ready to go into public beta and would like to settle on storage now so I don't have to move TB's of data later while in production. I think the s3 api might be a bit more work to implement, but it might not be too bad if its just for reading/writing media files.
Any advice appreciated
updated by @dannya: 01/06/15 11:50:32PM