Beautiful Data I Resources > Tools

Day-to-Day Tools

Beautiful Data participants self-organized a session to compare notes and share advice on the tools they frequently use in their work. Here are some of their recommendations for day-to-day tools: Browsers:

  • VirtualBox (modern.ie) is useful for testing things out on various browsers.
Note-taking:
  • Evernote offers many levels of organization and has a pretty robust free version.
  • Notability (iPad app) allows you take notes by hand while using cloud storage and syncing your notes with a range of programs. It can save clean PDFs as well as PDFs that include elements like highlighting and annotations.
Project management:
  • Dropbox is popular because of the ease with which you can move things between places and devices.
  • Basecamp is helpful for managing collaborative projects.
  • Flow is an alternative to Basecamp that some find more visually appealing.
  • Zotero is generally useful for citations and research projects.
Bookmarks and archiving:
  • Delicious is a popular social bookmarking tool that can can link with Twitter (but doesn't have to) and offers a strong tagging system.
  • Pinboard is pitched as “social bookmarking for introverts” and enables you to tag things and link to various tools and social media platforms. For an additional $25/year, it will archive the full text of anything you save.
  • Pocket and Instapaper are tools that help you collect material, save pages, and archive things so that you can read them later.
Text editors for large amounts of text:
  • Open Refine is useful for working with messy data. You can load databases/spreadsheets, or comma/tag-separated text, and it generally cleans it up pretty instantly.
  • Sublime text 2 is a text editor "for code, markup, and prose," and one nice feature is its ability to search and replace by regular expression.
  • Notepad++/NoteTab are other text editors that may be useful.
Writing tools:
  • Scrivener is useful for writing things that aren’t “real drafts” (it has a "scratch paper"-like feel and lets you move things around easily and pin images on cork board).
  • Ulysses 3 (Mac-only) is particularly helpful if you write primarily for the web, because it is plain text-based and uses markdown.
Scanning:
  • The iPhone is relatively good scanner, particularly in combination with the app PDFPen Scan +, which does scanning, OCRs on phone, and embeds this in metadata, so it's all there when upload; you can copy text straight from the app.
  • Camscanner app can do some of this if you have Android, not iOS.
  • VueScan is pitched as “the software that should have shipped with your scanner” and supports a huge list of scanners.
Storage management:
  • Time Machine is the default option for incremental backups on Macs.
  • Many people use Dropbox and Google as storage in addition to their hard drives.
  • Backblaze is a cloud-based option for backup, and it automatically sends changes in files to cloud. It costs $5/month.
  • Carbon Copy Cloner is useful for helping solve issues quickly if something breaks.
  • Some general rules: “expect migration” and use 3-2-1 backup: 3 copies, 2 different media, 1 off-site.

Visualization tools

The list below is a small collection that was put together for the workshop, but there are many many more great tools out there. Visualizing Advocacy has a fantastic guide that groups tools by function and includes in-depth reviews for many. DataVisualization.ch has another great (searchable) collection, primarily consisting of Javascript libraries.

Data Scraping

Import.io Desktop app creates a custom API to grab data from any existing webpage page

Data Cleaning

Google Fusion Tables Online tool used with google drive for data cleaning, filtering, combining, and basic visualization including mapping. Can import csv, tsv, txt. Easily integrated with Google Spreadsheets. * geocodes location information * no coding required Open Refine Open source desktop app, can import csv, json, rdf, tsv, xlsx, xml, google spreadsheets. Primarily for cleaning, filtering, and exploring existing datasets. Data Wrangler Online tool that takes existing csv files and allows you to edit dataset. * no coding required, but steeper learning curve Data Converter Converts Excel to web-friendly formats (e.g. JSON, XML)3.

Data Mapping

TileMill Desktop app for mapping and styling geolocated data. * style maps with Carto (CSS-like) * Can publish interactive maps to the web with MapBox (an open source mapping platform) Google Maps Engine

Data Visualization

Non-coding Raw - Web app for creating custom vector-based data visualizations. Slows down with big datasets. Built on D3 (see below). Timeline.js - Timeline tool. Imports data from google spreadsheets. Can be embedded as iframe. Easily integrated with many popular media platforms (YouTube, Flickr, SoundCloud, etc.) Tiki Toki - Similar to Timeline.js with more visual customization. Variance charts - Javascript library that creates html/css driven visualizations. Google charts TimeFlow - Open source desktop app, can import Excel, CSV, or TSV files. Coding (mostly Javascript) D3 - Powerful Javascript library for interactive visualizations and data manipulation. Many examples and tutorials available. Nester is a tool for experimenting with D3 and nesting data. Gephi - Desktop app for network mapping Processing / Processing.js Recline.js - Javascript library for building data applications. Tangle.js - Javascript library for creating reactive documents. Paper.js - Open source vector scripting that runs on HTML5 canvas. Mapbox - Open source Javascript library for creating interactive maps, works with Tilemill. Modest Maps - A simpler Javascript library for creating interactive maps.