Big Data for a Big City

Posted by:

Jackson Hts., New York, September 9, 2014 - Big Data smells Orwellian. And we might come to hate it some day. But today it’s everywhere. Sensing power and profits corporations and governments corralled the smartest graduates to figure out ways to extract value from the data ocean that’s become available through cheap technology and social media.

With .nyc’s activation another big data layer is becoming available from 2 data logs created by user searches. One is a Data Query Log that records successful searches of the .nyc TLD. Perhaps more interesting is the Error Log, which records unsuccessful inquiries. With an increasingly intuitive web, we can expect more people to take a risk and directly type-in desired domain names, rather than relying on Google search for their every need. Type-ins not reaching an existing website will end up in the Error Log. We’ve provided some thoughts on possible uses of the DNS search data, e.g., imagine creating a “City Pulse” comprised of 311, tweet, and the DNS Data Logs. We’ve elaborated on these prospects on our DNS Data Query Log wiki page.

One traditional problem with releasing this type of data relates to data mining, called front running in the domain name industry. Some see an unfair advantage arising from someone searching the Error Log for insights into domain names worth purchasing. Indeed, some might make a career of watching the error logs and registering names. But with an effective nexus policy, we look forward to the local jobs created by a frontrunner marketplace.

The city has yet to decide on a policy for releasing this data. We’ve advocated for its release within a framework of data privacy standards and clear and effective controls. (Commons image courtesy of Thierry Gregorius.)

0