VizHub 1.0
This post summarizes changes and new features in VizHub, a new online platform for teaching & learning Data Visualization with D3.js & JavaScript, since its Beta Release in August.
VizHub has been put through its paces as the content creation and assignment submission tool for Data Visualization Course 2018, a 10-week online graduate course at WPI. The development work since the beta release has been driven primarily by real user feedback, mostly from the students in the course. I’m really excited to see the platform harden into a usable product, and to see its effectiveness as a teaching & learning tool in practice.
By the way today is my 2⁵ birthday! If you appreciate my work in creating public and Open Source content (or just want to give a birthday gift), please consider becoming a patron and donating $1/month.
What is VizHub?
To summarize from the older posts VizHub Alpha and VizHub Beta, I created VizHub primarily as a teaching tool. When I started teaching courses that involved coding years ago, I would start with desktop-based tools. Frustrated by the fact that it took students weeks to learn everything required to set up their local development environment (and Git/GitHub to boot), I started using online code editors.
At first (circa 2010) I used the venerable JSBin. This worked quite well for years, but I wanted to also give students the ability to publish their work more “properly”. While JSBin focuses on delivering a solid editor for HTML, JS and CSS, its focus is not on presenting the work. I really wanted to have students use bl.ocks.org, because works published there are published “properly”, with the running program out front, the author’s name, date, and a description.
When the excellent project Blockbuilder came out in 2015, I was thrilled that now students could easily publish works to bl.ocks.org. The barrier of learning Git and local development was lifted. I used Blockbuilder to teach last year’s online Data Visualization course with WPI, and it worked out pretty well. The main frustration was in developing more complex programs. When I (or students) wanted to reach for ES6 modules, they were not available to use in Blockbuilder. I wanted to use a system similar to Blockbuilder, but with support for ES6 modules.
CodeSandbox came pretty close to making the cut, as you can use ES6 modules there. However like JSBin, there is no design towards presentation of the work. Also, as in desktop-based development, CodeSandbox exposes the complexity of the underlying tools (e.g. package.json, Webpack, Parcel).
Observable was another strong contender, but I didn’t want to use it for teaching because there are too many nonstandard platform-specific things students would need to learn in addition to the already complex Web standards (and D3). The real show-stopper for me with using Observable for teaching is you can’t export work to develop it further outside of Observable.
That’s not to say you can’t export your code from Observable — you totally can! What I mean to say here is that you can’t comfortably further develop exported Observable notebooks outside of the Observable platform. For comfortable further development, you must go back into the Observable platform.
This is fine for many cases, and it’s really amazing to see the thriving creative community spawned within Observable. I’d encourage you to try out Observable and peruse it’s recently published works (which I do check often). It’s really cool, but the platform lock-in aspect is a show stopper for my particular use case. Namely, teaching data visualization using today’s standards, for application in any environment, especially for export and integration into ongoing projects (e.g. proprietary, company projects) that already use ES6 modules.
Over the years I’ve had many ideas for improvements to bl.ocks.org such as real-time collaboration, view count, forks linkages, dataset linkages, embedding, a comments section, drop-shipping prints, and more. Rather than use an existing system, I decided to go ahead and build the system that I wanted to use for teaching data visualization. The core idea is to make it simple and easy to develop visualizations in the browser using standard technologies, then build out the platform over the years to be more of a visualization publishing and collaboration platform. VizHub was born.
The following sections describe new features added since the Beta Release.
Visualization Export
One feature that quickly bubbled to the top of our upvote-prioritized backlog was visualization export. This issue received 5 upvotes within the first week after the Beta launch, whereas there were hardly any upvotes on other issues. This is indeed a “core feature”, as one goal of VizHub is to enable users to create things using vanilla technologies such as ES6 modules, and not rely on any platform-specific things that would result in platform lock-in.
Clicking the “export” button will download a .zip file whose contents you can deploy on any static file server. These files also contain a package.json and rollup.config.js , so that you can edit and build the source files locally by running npm install && npm run build. This re-creates the Rollup-based build step used internally by VizHub in your local environment. Since the modules are based on the ES6 standard, exported projects should “just work” with Webpack or Babel as well, so it should be straightforward to migrate any code from VizHub directly into a real-world project.
The exported files are also compatible with bl.ocks.org. You can mirror your work from VizHub to bl.ocks.org using Gistup to publish the unzipped files. Another feature idea is to automatically mirror visualizations to bl.ocks.org, and keep them up-to-date as the visualization changes in VizHub. This would essentially make VizHub an editor for bl.ocks.org. If you like this idea, please upvote the Mirror to Bl.ocks.org issue!
Resizable Split Pane
Whenever I showed people VizHub and watched them use it, without fail they would try to resize the split pane between the code and the running program. Frustration ensued when there was “nothing to grab onto” in between the left and right panels. Since this would likely be one of the first things anyone tries, the split pane was made resizable (thanks to react-split-pane). In conjunction with browser zoom, this feature gives users full control over the balance of screen real estate between code and output.
Responsive YouTube Embeds
As more visualizations were created with YouTube embeds, the fixed width of embedded videos became a pain point. The videos were getting cut off on the right side, and when the visualization pane was large the videos only filled half the available space. Some special logic was added to make YouTube embeds responsive, so they resize gracefully as you resize the split pane, always occupying the full width available. Hopefully this makes it tolerable to watch the videos within VizHub, and follow along by manipulating the code while the video plays. Several students said this is their preferred method for learning.
Clarity on Licensing
Several folks inquired if they may use the code examples found VizHub, with some trepidation. My response was “Yes of course!”, at which point I realized that the licensing should be made explicitly clear to anyone viewing the works within VizHub. Therefore the following text was added to the bottom of the visualization view, for all visualizations:
All code in VizHub is released under the MIT License.
The goal of this policy is to make VizHub a place where folks can freely draw content from for use in real-world (proprietary) projects. This also makes the practice of forking & modifying “stress free”, encouraging creation of derivative works. I’m not sure if it’s critical to enable users to change the license on their works. If you have an opinion on this, please upvote or comment on the “Editable License” issue.
Customizable Height
Now you can choose the height of the visualization, like an artist can choose the size of her canvas. Just click “set height” and enter the new height in pixels. No longer are visualization authors confined to the limits of 960 X 500! Historical note: the default dimensions of 960 X 500 were chosen for compatibility with bl.ocks.org, which uses the same default dimensions.
Better Debugging
For the WPI Data Visualization Course 2018, I had been asking students to do their homework assignments using VizHub. For the first few weeks, I had made the assignments trivial, of the “fork & modify” variety, but in weeks 5 and 6 the assignments were more challenging, requiring original development work. When I asked students in week 6 to “voice their frustrations” about the course in general, much to my surprise the number one thing was “debugging in VizHub”.
Several students mentioned “hours of time lost” searching for a misplaced comma or syntax error, “unclear error messages”, “nonsensical line numbers”, and “bundle.js is not updating and I don’t know why”. This feedback made it crystal clear that improved debugging should be the top priority, as it’s causing the most pain, and inspiring the desire to “eject from VizHub”.
Inline sourcemaps were implemented shortly after receiving this feedback (thanks to rollup-plugin-hypothetical), enabling developer tools to point to the correct line numbers in the original ES6 modules, rather than the compiled bundle. Another change that surfaces syntax errors (and any other errors from Rollup such as mismatched import and export names, or importing from non-existent files) was also made.
Implementation of improved debugging support was unexpectedly very challenging, because there are so many kinds of errors that could happen. Runtime errors, syntax errors, importing from the wrong file, mismatched imports/exports — each of these required some special attention to get working. It was like a good game of “Whack-a-mole”, where solving one case would break others. Luckily the students posted questions about “bundle.js not being updated”, and shared links to programs in VizHub that reproduced the behavior, so I could use their examples as test cases to solve, one by one.
Delete Visualization
Now you can delete visualizations. This was the number-one upvoted feature, accumulating 11 upvotes! This is the last of the issues that have had more than 1 upvote. It seems the upvoting model has stagnated after the previous Beta launch. If you want to help prioritize development, please have a look at the public backlog and upvote the issues that look interesting to you!
More Complete Text Editor
One of my students said “Part of the reason I would like to develop outside of VizHub is to be able to use a more complete text editor.” This struck me as a totally valid point, and also one that can actually be addressed. Thankfully, the mature text editor library CodeMirror is being used already, and the CodeMirror ecosystem has plenty of “add-ons” for adding functionality. To make the editor more complete, SublimeText bindings were integrated into the editor. These include a wide range of features such as search & replace, collapsing sections, and matching brackets.
Then I got thinking, what could be done to make the VizHub editor experience even better than full-featured traditional IDEs? What if VizHub could deliver something above and beyond, that would make people want to migrate from a desktop editor to VizHub, not the other way around?
Then I remembered the excellent project Inlet, which gives an interactive slider for numbers, checkbox for booleans, and color picker for colors (inspired by Bret Victor’s amazing talk “Inventing on Principle”). This is implemented in Blockbuilder to great effect. Considering it low hanging fruit as the integration is simple to implement, Inlet was also added to the editor.
The Road Ahead
So what’s next for VizHub? You tell me! Please upvote the issues that are important to you in our public backlog (which reorders based on upvotes). For a snapshot of the development state, see our Scrum board. This is actually used, and updated as things bubble up in priority and are worked on.
In terms of VizHub features, the rough prioritization at the moment (for the large features) is roughly as follows: Thumbnails, JSON Datasets, Forks links, Mirror to Bl.ocks.org, JSX Support, Embedding, Comments, View Count, Real-time Collaboration, Statefulness, and Composition.
On the content front, I’d like to make videos and implementations for: Line Chart with Multiple Lines, Stacked Area Chart, StreamGraph, Stacked & Grouped Bar Charts, Bubble Map, Using a Voronoi Overlay, Pie Chart, Donut Chart, Coxcomb Plot, Pan & Zoom on a Globe, Labels on a Map, Treemap, Sunburst, Icicle, Force Directed Graph, Small Multiples, Displaying Multiple Views, Linked Navigation, Bird’s Eye Map, Brushing, Linked Highlighting, Aggregation (Histogram, Group-By), Hexbinning, Crossfiltering, Parallel Coordinates, Box Plot, Responding to Resize, Redux with D3, React with D3, and then a whole series on optimizing interactive visualizations with Topologica.js (a library that implements reactive data flow).
Give Me Money
The online course ends November 1st, and with it the funding that has supported this work thus far. Even after delivering most of the content for the online course, I feel like I’ve just scratched the surface of what’s possible with this medium of online teaching and Web-based data visualization authorship.
What I would really like to do is keep going forever creating Open Source interactive data visualizations, and also developing VizHub into a full fledged online platform for creating, publishing, critiquing, and collaborating on data visualizations. I want to push the boundaries of what’s possible in terms of scaling complexity of interactive visualizations, and by making videos of the work I want to bring others along for the ride, enabling them to create visualizations of their own.
I’ve noticed an interesting phenomenon with the students in the course, namely that after I create a video (and implementation) of a certain topic or technique, it enters the “realm of possibility” for them to make use of, without fear and trepidation. They start to think with that new concept in their toolbox. For example, I was sort of surprised that the student project proposals included techniques I had taught, but not techniques I had linked to examples of “in the wild” (on bl.ocks.org and elsewhere).
I’d like to continue doing this sort of work and cover the design space of interactive visualizations, essentially conducting a course “at scale”. The goal is to unlock (bring into the realm of possibility) a variety of topics and techniques for anyone who cares to view and study the public content. The main audience for this would be the existing D3 and Data Visualization communities, and also newcomers to data visualization who want to expand their capabilities.
In order to make this dream a reality, I need your help. Financially. I currently need to spend most of my time working on consulting projects to pay the bills. While this is fine, I’d prefer to spend most of my time on these passion projects — creating Open Source visualizations, creating videos, and evolving VizHub. With your help, this is entirely possible!
If you appreciate this work and want to support me in this quest, please consider becoming a patron on Patreon. With a $30/month contribution, you can join an exclusive online learning community, hosted by me and with no university affiliation, where we will conduct somewhat of an indefinitely ongoing “Data Visualization Masterclass” in which students can interact with me directly, share their work with peers, and ask for constructive critique.
Thanks for reading!
Related links:
- Data Visualization Course 2018
- VizHub in GitHub (Scrum Board, Upvote-sorted Backlog)
- D3 Community Slack (join the #vizhub channel there)
