Over at Fell in Love with Data, Enciro has a great interview with Processing guru Miriah Meyer. In the interview they get into some of the best and worst features of the tool, and provide some great insight into how to get started with Processing and what you can expect to get from it.
In short, the best aspect of Processing is the amount of code it takes to get a simple scene with callbacks going — it is a small fraction of what it would take with OpenGL. Simple primitives like circles, squares, text, etc. are nicely abstracted into one-line function calls. Mouse and keyboard callbacks are automatically handled. There is a wide variety of common graphics helper functions available, like lerp-ing colors. Full-screen apps work without having to grab weird OS handles.
Jim Blackhurst has a nice writeup on his blog about working with large SQL-based data in Processing and OpenGL for creation of big point clouds and heatmaps.
I’ve been using Processing to create tools that render the heatmaps, but while the logical structure of program is fairly simple, there are significant challenges in working with large datasets. The primary challenge is loading the data into memory. The data is all held in a SQL database, and while I could connect to the DB directly using processing, the DB is optimised for data-in operations, not data-out, so you don’t want to be pulling the data out too often. Instead, I dump the raw spatial data (X,Y,Z coordinates) into a CSV file, one record per row. I usually create heatmaps from datasets in excess of 1 million rows, and most of them are between 5 – 20 million rows (I have one that is 22 million rows!). A CSV file containing 10 million rows of spatial data is about 364MiB in size (the 22.3m row CSV is 802MiB!). In order to create the data structures in memory to hold sets this large, I have to work in 64bit mode to get over the Windows 32bit memory restrictions.
The dataset is a collection of 11.3 Million Player deaths from the game “Just Cause 2″, hopefully showing the most dangerous areas of the game. The project is still in-development and he hopes to transition to the new Deus Ex game for his next effort.
If you’ve ever wanted to learn the visualization language “Processing”, now’s your chance. Every day during the month of May, the Vormplus blog has entries for “Processing Month”, covering everything from the mundane (creating points, drawing lines) to the advanced (Data queries and interactivity). Looks like something to bookmark!
If you’ll be in Brooklyn in two weeks, consider checking out this Intermediate Processing Workshop at the Devotion Gallery. The workshop 4 days, 4 hours each day, with a registration of $350 ($250 for Students).
This four-day workshop will introduce the participants to Processing syntax and coding structures, going from the basics to more advanced concepts like object-oriented programming and using libraries to extend Processing’s core functions. Participants will receive a pre-prepared library of code examples demonstrating the topics covered in the workshop, and will have a chance to discuss how they might use Processing in their existing practice
Jonathan Feinberg has just released on GitHub a clever Python-like wrapper for Processing, the result of an interesting encounter with a group of 8th graders.
I recently gave a talk about Processing to a group of rather bright 8th-graders, as part of a computer-programming summer camp they were attending at my office. Their curriculum up to that point had been in Python, which is an eminently sensible choice, given the pedagogical roots of the language.
The kids were really turned on by the demos–I showed them the white glove, and Golan Levin’s New Year’s cards–but they were bogged down by Processing’s C-like syntax, which really seems arcane and unnecessarily complex when you’re used to Python.
The full code and some demos are available at GitHub.
StreamGraphs are probably one of the most visually impressive visualization algorithms out on the web, made popular from the New York Time’s “Ebb and Flow of Movies” several years ago. While the algorithms have been available open-source for Processing for a while now, it did require Processing and Java to use them. Now, you can try them out in just JavaScript thanks to the efforts of Jason Sundram who has uses the Processing.js code to create Streamgraph.js, and even added a few new features:
Dynamic sizing (sized to fit the Canvas element it’s on)
Settings. These can be dynamically changed, and easily set outside of the streamgraph code.
Zoom. You can click and drag to zoom in, and press Escape to zoom out.
Hover. The selected layer changes color, and the active layer name and data point is displayed at the top of the screen.
Another color option that offers a bit more variation.
Examples (including the original beautiful examples).
It isn’t quite up to the New York Times version, but still impressive.
A beautiful experimental painting program, Peter Blaskovic’s “Flame Painter” website is a java applet that combines chaotic curves with some beautiful color blends to create some beautiful pseudo-random art with your mouse.
Flame is a painting program, it belongs to my “I am Artist” experimental project. I think with tools which inspires you, everybody can be an artist. You can try it here, change different brush settings and paint your own flame paintings. When you change background from black to white, palette changes from additive to substactive and the feeling of the painting is very different. It’s not easy to explain all brush parameters, so I leave this for your experimentation.
Definately worth spending a few minutes playing with.
Micheal Ogawa at UC Davis has used Processing to build a fascinating visualization tool for source code repositories. His explanation is pretty good:
This visualization, called code_swarm, shows the history of commits in a software project. A commit happens when a developer makes changes to the code or documents and transfers them into the central project repository. Both developers and files are represented as moving elements. When a developer commits a file, it lights up and flies towards that developer. Files are colored according to their purpose, such as whether they are source code or a document. If files or developers have not been active for a while, they will fade away. A histogram at the bottom keeps a reminder of what has come before.
It uses organic information visualization methods, which results in an eerie movie of something akin to cell mitosis or bubbles as the various files coalesce and fly apart. Look after the break for a video of his algorithm applied to the development of Eclipse, but be sure to his his side for videos of various OSS projects like Python, PostgreSQL, and Apache on his side.
Our recommendation for this week goes out to anyone interested in getting their feet wet in information visualization via the Processing Language. Ben Fry, an MIT Graduate who helped build the Processing language, has a great O’Reilly book entitled “Visualizing Data: Exploring and Explaining Data with the Processing Environment“. The book description:
This book teaches you how to design entire interfaces around large, complex data sets with the help of a powerful new design and prototyping tool called “Processing.” Used by many researchers and companies to convey specific data in a clear and understandable manner, the Processing beta is available free. With this tool and Visualizing Data as a guide, you’ll learn basic visualization principles, how to choose the right kind of display for your purposes, and how to provide interactive features that will bring users to your site over and over. This book teaches you:
The seven stages of visualizing data — acquire, parse, filter, mine, represent, refine, and interact
How all data problems begin with a question and end with a narrative construct that provides a clear answer without extraneous details
Several example projects with the code to make them work
Positive and negative points of each representation discussed.
The focus is on customization so that each one best suits what you want to convey about your data set. The book does not provide ready-made “visualizations” that can be plugged into any data set. Instead, with chapters divided by types of data rather than types of display, you’ll learn how each visualization conveys the unique properties of the data it represents — why the data was collected, what’s interesting about it, and what stories it can tell. Visualizing Data teaches you how to answer questions, not simply display information.
Who knows, for ther mere $26 price tag you could become the next Aaron Koblin.
A fascinating interactive table display allows users to interact with a pair of physical objects on a large table-display to view fonts. It has a huge amount of new interaction technologies, user interface metaphors, and just general awesome-ness.
The interactive table is an implementation of the reacTIVision system and was built by students and staff from the FHP for the mæve installation and we are really really grateful for the opportunity to use this table as a basis for our work.
The application running on the table is written in Java (using the NetBeans IDE) and based on the Processing framework. We use a MySQL Server for our database and the Processing SQLibrary to connect to it. The communication with the table runs over the TUIO protocol and uses the Processing TUIO Client. Mathematik – a part of the gestalt framework was very helpful for vector math and we have to thank Rob van der Woude for his PrintPDF batchfile.
See a video of the system in use, after the break.
Over at Fell in Love with Data, Enciro has a great interview with Processing guru Miriah Meyer. In the interview they get into some of the best and worst features of the tool, and provide some great insight into how to get started with Processing and what you can expect to get from it.
Jim Blackhurst has a nice writeup on his blog about working with large SQL-based data in Processing and OpenGL for creation of big point clouds and heatmaps.
If you’ve ever wanted to learn the visualization language “Processing”, now’s your chance. Every day during the month of May, the Vormplus blog has entries for “Processing Month”, covering everything from the mundane (creating points, drawing lines) to the advanced (Data queries and interactivity). Looks like something to bookmark!
If you’ll be in Brooklyn in two weeks, consider checking out this Intermediate Processing Workshop at the Devotion Gallery. The workshop 4 days, 4 hours each day, with a registration of $350 ($250 for Students).
VizWorld.com 
Comments