Data Wrangling

The objective of this assignment is to expose you to some issues of acquiring data. First, we will examine some of the different formats that are used with data files. Oftentimes you can select the format that you will work with, but sometimes systems will force you to accept set data formats. Second, you will see firsthand how data are often provided in either an untidy format or just downright “dirty.” Working with data can be messy, it requires a careful, observant, and methodical approach.

This assignment has two parts: (1) Multiple Formats – why are there different formats? You will work with RDF, XML, JSON, and CSV formats. (2) Identifying Issues with a dataset and developing a strategy to manage/correct issues.

Part 1: Multiple Formats

Download the four zipcode Demographics files. These files contain the same data but in different formats. Using a text editor examine the files and note differences in their structure. Perform some research to obtain brief definitions of each format, write your definitions stating your referenced source(s). Is any format easier to read compared to the others? If so, why?

Examine the file and characterize it (i.e., pick a format to analyze). Imagine that you must present your findings to an executive committee that wants to know what is important in the data. Prepare a single PowerPoint slide to succinctly characterize the data.

Part 2: Data Wrangling

Data wrangling is the process of manipulating data to change it into a format that can be used for analysis so that the data can be used in automated/computerized analysis tools. There are two major stages: data tidying and data cleaning. Data tidying is the process of transforming data so that it is in a format where it can be readily processed by automated tools. Data cleaning is the process of transforming the data to insure to the greatest extent possible that it accurately reflects the subject that it pertains to.