From Gunrock to JSON

How do we export information from Gunrock?

Typical programs use "printf" to emit a bunch of unstructured information. As the program gets more sophisticated, "printf" is augmented with command-line switches, perhaps a configuration file, but it's hard to easily parse random printf output.

More structured is JSON format. JSON is a nested dict (hash) data structure with arbitrary keys (and arbitrary nesting). It can be used to hold scalar, vector, and key-value data. Many tools can input and output JSON. It is a good choice for exporting information from a Gunrock program.

Ideally, we would declare a C++ struct or class and simply print it to stdout. The particular issue with C++, however, is that it poorly supports introspection: a running C++ executable does not know anything about the internals of the program that created it. Specifically, it doesn't know its own variable names, at least not without an incredible amount of pain. Maintaining a set of strings that map to variable names is undesirable since that can get out of sync.

Instead, we've elected to use a dict data structure that stores the JSON data, and we will write directly into it. We are using a header-only JSON generator based on Boost Spirit. It's used like this:

json_spirit::mObject info;
info["engine"] = "Gunrock";

Currently we can output JSON data in one of three ways, controlled from the command line:

The current "automatically-uniquely-named file" producer creates name_dataset_time.json. By design, the file name should not matter, so long as it is unique (and thus doesn't stomp on other files in the same directory when it's written). No program or person should rely on the contents of file names.

The current JSON structure (info) is passed by reference between various routines. Yuechao suggests that putting info into the global Test_Parameter is a better idea, where it can be passed into the enactor's and problem's Init() routines.

We don't have a fixed schema (yet), so what's below reflects what we put into the test_bfs code. Some of these are likely not useful for any analysis, but it's preferable to include too much info in the JSON output rather than not enough.

Fields that should be in any Gunrock run

Fields for any traversal-based primitive

BFS-specific fields

Thread safety: "Using JSON Spirit with Multiple Threads"

"If you intend to use JSON Spirit in more than one thread, you will need to uncomment the following line near the top of json_spirit_reader.cpp.

"//#define BOOST_SPIRIT_THREADSAFE"

"In this case, Boost Spirit will require you to link against Boost Threads."

link

If compilation is too slow

Currently we're using the header-only version of JSON Spirit, which is easier to integrate but requires more compilation. The (docs)[http://www.codeproject.com/KB/recipes/JSON_Spirit.aspx#reduc] have ways to increase compilation speed.