How to store files in Cassandra with NodeJs?

Let’s learn about how to store files in Cassandra using NodeJs. In Cassandra, a blob data type is capable of storing images, videos or other file formats (pdf, xlsx, etc.)

This POC covers the following:
Connectivity with Cassandra in NodeJs.
Part I : To insert and fetch an image using NodeJs and Cassandra?
Part II : [For large files]To insert an image in chunks, fetch those chunks and display the image.

Find full code on Github : https://github.com/crazyrich/NodeJS-Cassandra

Part I: To insert and fetch an image using NodeJs and Cassandra?

UploadFile is our request handler in which we first read the file using readFileSync with  base64 encoding and then wrap it into a Buffer. Buffer is equivalent to Blob in Cassandra. We execute the query and insert this buffer into a blob column in Cassandra table. Blob will store the content in some hexadecimal values (eg. 0x8678298439..). Create files folder at the root level to keep the images (which we will upload).

In GetFile, we hit the Select query to get the blob stored in Cassandra table and write it into a file. After execution, you may check the new image created at the root folder.

This approach is OK when your files are smaller in size(KBs).

Part II: Insert an image in chunks, fetch those chunks and display the image.

For images of varying size (in MBs), it’s advisable to first break the images in chunks and then store them in your table. So using createReadStream, we create a readstream for our file and check for the ‘data’ event. CreateReadStream has attributes Encoding type and HighWaterMark. HighWaterMark is used to define the chunk size. Here we have defined 128 *1024  = 131KB which means a chunk can have a max of 131KB size. As soon as a chunk is read, this event handler wraps it into a Buffer and inserts it into the table. This is done for all the other chunks created.

In GetFileChunks, we select all the chunks against an object and merge them in order into 1 Buffer and write that Buffer into a file using writeFile (from ‘fs’ module). After execution, you may check that the file has been created at the root level.

This was all about storing files or store images/media in Cassandra with NodeJs.

Hope it was helpful. Thanks!

Leave a Comment