Category: geek-instructions

One of my “projects” was to be able to store all my data and be able to add information to it. I gave it some crazy name, and let it sit and pickle away in my mind until it matured.

In comes .meta-json. With every file that would be stored (such as file_name.ext) , there would exist a separate file stored alongside that original file that would have the additional extension of .meta-json.

.meta-json is meant to be a common, easily indexable and searchable format that gives files a raison d’être. the format is rooted in json with specifics that helps to define the metadata of the file.

The proposed format comes down to:

{
   'author': 'unknown',
   'type': 'image',
   'title': '',
   'description': {
      'en': 'something'
   },
   'date': '',
   'source': '',
   'location': '',
   'people': [''],
   'checksum' {
      'sha256': 'xxxx'
   },
   'mimetype': '',
   'creation-date': '',
   'filesize': ''
}

This format pretty much breaks down to key/value pairs. Every key can either have a string or an array/object as a value, depending on the name of the key.

Additional fields can be defined and exist, but a majority of the data should be present in the metadata for each file to complete it.

This is the initial spec of the .meta-json format.

It’s been a long time since I had FUN with a programming language. Today I had decided to take some time and dig nose deep into node.js. I’ve written much with Javascript before, and have always wanted to create RESTful services with something that wasn’t PHP, and from what i’ve heard node.js is a nice transition for a developer wanting to learn a new thing.

So, step 1 was to get some revision loaded up. Instead of apt-get or yum installing myself a version, I decided to get v8.11.3 (at this time is LTS) and install it in /usr/share/node. I symlinked all necessary executables to /usr/bin, and node -v’d myself to satisfactory execution.

I did the obligatory hello-world.js, and a subsequent cURL later to localhost:3000 got me really tickled. Being familiar with Javascript this was a confusing walk in the park. I decided to go ahead and github/public my musings and ramblings and set some goals, as I do have a destination with this.

https://github.com/mjheick/nodejs-learnings

002-is-global.js: First thing was to find out if I have access to global variables. Yes. Yes I do. Thanks Javascript for still allowing this to happen.

003-what-is-req-and-res.js: Then, with this res and req, I had to find out the contents of them. There had to be a Data::Dumper or var_dump somehow, and (since Javascript) there was the trusty console.log() to inspect these variables. #awesome

004-rest-verbs.js: Cause we’re building something RESTful, I needed to think if I had access to some Verbs. Low and behold, after skimming the node.js api for http the .method was smack in my face. It really came down to RTFM.

005-list-add-remove.js: I just took off from there. We have access to a global, we can make logic decision based on methods passed, and now we can pass a querystring and parse it up! At this point things were looking fairly awesome.

006-post-list-add-remove.js: We can’t use querystring for long, so I had to convert it to data passed in the request. After reading how Events and EventEmitter worked, this seemed quite natural. Got into some try/catching as well, and now explicitly taking in JSON data as well. Just a couple steps away from what we need to actually do!

007-list-with-expired-items.js: At this point we needed to store data with some “life” tied to it. i had to kick in some setInterval logic to scan the list of items that we’re storing and cull any that have expired. I also approached a level of pride in my code by making it look more presentable. At this point of excitement I had decided to develop a dataloader to “test” how well it can load and purge data, as well as overall node.js performance.

008-persistent-executed-list.js: In loading node.js and terminating it I had to think ahead in “what to do if the process bombs out with data stored in it”. At this point I had plugged in two separate items:
A) process.on(), to hook in synchronous routines that saved data and performed any cleanup, and rightfully aborted the process.
B) fs.readFile() fs.writeFile, so that I can dump the “local memory storage” timely to disk in 30 seconds interval.

Overall, this was awesome. I wanted to be able to create a RESTful service quick, timely, and efficiently, with little “new knowledge” required from previous Javascript development, and this was exactly what I got.