Core Java

FileSystemMap: A Natural Way to Interact with your File System

As part of a project I’m working on at the moment I’ve been looking at creating a FileSystemMap. I’ve started a very small GitHub project here to host the code.

Essentially this map implementation is will allow the user to interact with a directory on their file system as if it were a java.util.Map. Each entry in the map will be a file in that directory, the key will be the file name and the value will be the contents of the file.

This code builds a FileServiceMap and adds five entries:

Map map = new FileSystemMap("/tmp/filetests");
  map.put("one", "one");
  map.put("two", "two");
  map.put("three", "three");
  map.put("four", "four");
  map.put("five", "five");

This results in a directly structure like this:

/tmp/filetests/

|----- five

|----- four

|----- one

|----- three

|----- two

Adding and removing entries will change the files in your directory. Changing the value of the entry will cause the file to be re-written with the new value as its contents. For more examples see the code in testMapMethods.

Additionally the FileSystemMap has been designed for two way interaction. Any programmatic updates to it are reflected on the file system and any updates to the file system will be picked up by the map and fired as events.

This code registers changes to the file system and prints them out:

Map map = new FileSystemMap("/tmp/filetests");
map.registerForEvents(System.out::println);

This is some example output:

FPMEvent{eventType=NEW, programmatic=true, key='one', value='one'}

The eventType is one of:

  • NEW – a file has been created
  • UPDATE – a file has been modified
  • DELETE – a file has been deleted

The programmatic flag indicates whether it was the FileSystemMap itself that caused the event to be fired. e.g. if put() is called, a file will be created which in turn will cause an event to fired. To avoid feedback it can be useful to know whether it was an operation on the FileSystemMap that triggered the event.

The key is the name of the file that changed.

The value is the latest value associated with the file that has changed.  Note: this may or may not be the value that actually triggered the change.  For example if there were two very fast changes to the entry it is entirely possible that the value for the first event will pick up the value after the second update has already taken place. e.g.

map.put("one", "1");

map.put("one", "2");

could produce this output:

FPMEvent{eventType=NEW, programmatic=true, key='one', value='2'}

The first event (triggered by setting to “one” to ‘1’) is picked up but by the time the program has checked the contents of the file the file has changed to ‘2’. The second event is then picked up (triggered by setting to “one” to ‘2’) but this is suppressed because the value has not changed.

Some notes:

  1. Files starting with a ‘.’ are ignored. The reason for this is that many editors (including vi) create temporary files which should not be picked up by the FileServiceMap.
  2. If you look at the code you will notice that the WatchService (since Java7) was used to monitor changes to the file system. It is important to understand that the WatchService is OS specific. In particular it doesn’t work that well on Mac OSX. This thread discusses some of the issues.  Updates from the WatchService are slow and and fast flowing events can be dropped. In my testing Ubuntu performed significantly better than MacOSX.  However if you’re mainly interested in changes to the file system carried out by hand even Mac OSX will be fine.
  3. The Map only supports Strings.

It goes without saying that this class is designed for its specific utility rather than any sort of performance.

All contributions to this project are welcome!

Daniel Shaya

Daniel has been programming in Java since it was in beta. Working predominantly in the finance industry he has created real time trading and margin risk applications. He is currently a director at OpenHFT where we are building next generation Java low latency products.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Milky Floor
Milky Floor
1 year ago

It’s always been hard for me to figure out how to create file systems, as it’s much easier to use out-of-the-box options. I remember 7 years ago I tried to create my own file hosting, but it didn’t last more than a couple of months because I couldn’t support it. That’s why now I prefer anonymous file sharing on reliable hosts.

MarkUltra
MarkUltra
1 year ago

It’s always been hard for me to figure out how to create file systems, as it’s much easier to use out-of-the-box options. I remember 7 years ago I tried to create my own file hosting, but it didn’t last more than a couple of months because I couldn’t support it. That’s why now I prefer anonymous file sharing on reliable hosts.

Back to top button