Tuesday, September 8, 2015

Portable API Driven Or At Least JSON Driven Interactive Visualization Tooling

As I am working on the API and JSON driven visualization strategy for my Adopta.Agency open data work, I saw cloud monitoring platform Librato, publish their new "Space" interface as a Heroku add-on. I like dashboards and visualization tooling that can live on multiple platforms, and engineered to be as portable and deployable as possible.

In a perfect world, infographics would be done using D3.js, and would all show their homework, with JSON or API definitions supporting any visualizations. All of my Adopta.Agency projects will eventually possess a simple, embeddable, D3.js visualization layer that can be published anywhere. Each project will have its JSON localized in the publicly available Github repository, and be explorable via any browser using Github Pages.

The Librato approach reminded me that I'd also like to see modular, containerized versions of more advanced tooling, dashboards, and visualizations around some projects. This would only apply in scenarios where a little more compute is needed behind the visualizations, that could be done with simple D3.js + JSON, hosted on Github. Essentially giving me two grades of portable visualization deployment: light and heavy duty. I like the idea that it could be a native add-on, whereever you are deploying an open API or dataset.

I still have a lot of work to do when it comes to the light duty blueprint of JSON + D3.js, and API + D3.js, to support Adopta.Agency. I will focus on this, but keep in mind doing modular cloud deployments using Docker and Heroku for the datasets that require more heavy data lifting.

from http://ift.tt/1UFBYUn

Saturday, September 5, 2015

Pushing Forward Algorithmic Transparency When It Comes To The Concept Of Surge Pricing

I've been fascinated by the idea of surge pricing, since Uber introduced the concept to me. I'm not interested in it because of what it will do for my business, I'm interested because of what it will do for / to business. Also I'm concerned what this will do the layers of our society who can't afford, and aren't able to keep up with this algorithmic meritocracy we are assembling.

While listening to my podcasts the other day, I learned that Gogo Inflight wifi also uses surge pricing, which is why some flights are more expensive than others. I long suspected they used some sort of algorithm for figuring out their pricing, because some flights I'm spending $10.00 for the flight, and others I m paying $30.00. Obviously they are in search the sweet spot, to make the most money off business travelers looking to get their fix.

Algorithmic transparency is something I'm very interested in, and something I feel APIs have a huge role to play in helping us make sense of just exactly how companies are structuring their cost structures. This is right up my alley, and something I will add to my monitoring, searching for stories that mention surge pricing, and startups who wield this publicly as part of their strategy, as well as those who work to keep it a secret.

This is where my research starts going beyond just APIs, but it is also an area I hope to influence with some API way of thinking. We'll see where it all goes, hopefully by tuning in early, I can help steer some thinking when it comes to businesses approaching surge pricing (or not). 

from http://ift.tt/1JI4ZK1

Being a Data Janitor and Cleaning Up Data Portability Vomit

As I work through the XML, tab & comma separated, and spreadsheet strewn landscape of federal government data as part of my Adopta.Agency work, I'm constantly reminded of how the data being published is often retribution, more than it is anything of actual use. Most of what I find, despite much of it being part of some sort of "open data" or "data portability" mandate is not actually meant to be usable by its publishers.

In between the cracks of my government open data work, I'm also dealing with the portability of my own digital legacy, and working to migrate exported Evernote notes into my system, as well as legacy Tweets from my Twitter archive download. While the communicated intent of these exports from Evernote and Twitter may be about data portability, like the government data, they really do not give a shit about you actually doing anything with the data.

The requests for software as a service providers, and government agencies to produce open data versions of our own user or public data, has upset the gate-keepers, resulting in what I see as passive aggressive data portability vomit--here you go, put that to use!! A president mandating that us database administrators open up our resources, and give up our power? Ha! The users who helped us grow into a successful startup, actually want a copy of their data? Ha! Fuck you! Take that!

This is why there is so much data janitorial work today, because many of us are playing the role of janitor in the elementary school that is information technology (IT), and constantly coming across the data portability vomit that the gatekeepers of legacy IT power structures (1st and 2nd graders), and the 2.0 silicon valley version (3rd and 4th graders), produce. You see, they don't actually want us to be successful, and this is one of the ways they protect the power they perceive they possess.

from http://ift.tt/1KxseMw

Sunday, August 16, 2015

Legacy Power and Control Contained Within The Acronym

As I wade through government, higher educational, and scientific research, exposing valuable data, and APIs, the single biggest area of friction I encounter is the acronym. Ironically this paradigm is also reflected in the mission of API Evangelist -- helping normal people understand what the hell an Application Programming Interface is. I live in a sort of tech purgatory, I am well aware of it. 

The number one reason acronyms are used I think, is purely because we are lazy. Secondarily though, I think there is also a lot of legacy power and control represented in every acronym. These little abbreviated nuggets can be the difference between you being in the club, or not. You either understand the technology at play, or you don't. You are in the right government circles, or not. You are trained in a specific field, or you are not. I don't think people consider what they wield when they use acronyms, I think there is a lot of baked in, subconscious things going on.

One of the most important aspects of the API journey in my opinion, is that you begin to unwind a lot of the code (pun intended) that has been laid down over the years of IT operation, government policy, and research cycles. When you begin to unwind this, and make available via intuitive URL endpoints, you increase the chances a piece of data, content, or other digital resource will get put to use--something not all parties are actually interested in. Historically IT, government, and researchers wield their power and control, but locking up valuable resources, playing gatekeeper of who is in, and who is out--APIs have the potential to unwind this legacy debt.

APIs do not decode these legacy corporate, government, and institutional pools of power and control by default. You can just as easily pay it forward with an API gateway, or via an API architect who sees no value in getting to know the resources they are putting to work, let alone it's consumer(s). However if done with the right approach, APIs can provide a rich toolbox that can assist any company, institution or government agency in decoding the legacy each has built up.

You can see this play out in the recent EPA, er I mean Environment Protection Agency work I did. Who would ever know that the EPA CERCLIS API, was actually the Comprehensive Environmental Response, Compensation, and Liability Information System API? You don't unless you are in the club, or you do the heavy lifting (clicking) to discover the fine print. I am not saying the person who named the Resource Conservation and Recovery Act Information API, the RCRAInfo service, were malicious in what they are doing--this type of unconscious behavior occurs all the time.

Ultimately I do not think there is a solution for this. Acronyms do provide us with a lot of benefit, when it comes to making language, and communication more efficient. However I think, just like we are seeing play out with algorithms, we need to be more mindful of the legacy we paying forward when we use acronyms, and make sure we are as transparent as possible by providing dictionaries, glossaries, and other tooling. 

At the very least, before you use an acronym, make sure your audience will not have to work extra hard to get up to speed, and do the heavy lifting required to reach as wide as possible audience as you possibly can. It is the API way. ;-)

from http://ift.tt/1fj97aj

Saturday, August 15, 2015

Asking For Help When I Needed To Better Understand The Accounting For US Federal Budget

As I was working my way through the data for the US federal budget, I noticed a special row in between the years 1976 and 1977. It simply had the entry TQ, and no other information available about what it was. 

To get an answer regarding what this entry was, I went to my Twitter followers:

Then, because I have the most amazing Twitter followers ever, got this response from Stephen H. Holden (@SteveHHolden):

When doing any open data work, you can't be afraid to just ask for help when you hit a wall. I've been doing data work for 25 years, and constantly hit walls when it comes to formatting, metadata, the data itself.

The moral of this story is use your Twitter followers, use your Facebook and LinkedIn followers, and make sure and publish questions as a Github issue--then always tell the story!

from http://ift.tt/1TGSWXl

Friday, August 14, 2015

Stepping Up My Open Data Work With Adopta.Agency, Thanks To Knight Foundation, @3Scale, and @APISpark

I always have numerous side project cooking. Occasionally I will submit these projects for potential grant funding. One of my projects which I called Federal Agency Dataset Adoption, was awarded a prototype grant from the Knight Foundation. It was the perfect time to get funding for my open data work, because it coincided with the Summer of APIs work I'm doing with Restlet, and work already in progress defining government open data and APIs with 3Scale.

After reviewing my Federal Agency Dataset Adoption work, I purchased a domain, and quickly got to work on my first two prototype projects. I'm calling the prototype Adopta.Agency, and kicking it off with two projects that reflect my passion for the project.

US Federal Budget
This is a project to make the US federal budget more machine readable, in hopes of building more meaningful tools on top of it. You can already access the historical budget via spreadsheets, but this project is work to make sure everything is available as CSV, JSON, as well as an active API.

VA Data Portal
This project is looking to move forward the conversation around VA data, making it more accessible as CSV and JSON files, and deploying simple APIs when I have the time. The VA needs help to make sure all of its vital assets are machine readable by default.

The first month of the project will be focused on defining the Adopta Blueprint for the project, by tackling projects that my partner in crime Audrey Watters (@audreywatters), and I feel are important, and set the right tone for the movement. Once the blueprint is stable, we ill be inviting other people into the mix, and tackle some new projects.

Adopta.Agency is not a new technology, or a new platform, it is an open blueprint that employs existing services like Github, and tools like CSV to JSON converter, to help move the government open data movement forward just one or two steps. The government is working hard, as we speak, to open up data, but these agencies don't always have the skills and resources to make sure these valuable public assets are ready for use in other websites, applications, analysis and visualizations--this is where we come in!

With Adopta.Agency, we are looking to define a Github enabled, open data and API fueled, human driven network that helps us truly realize the potential of open data and APIs in government -- please join in today.

from http://ift.tt/1DTylYx

Being The Change We Want To See In Open Government Data With Adopta.Agency

I have had a passion when it comes to open data for a number of years. Each time the federal budget has come out in the last 10 years, I would parse the PDFs, and generate XML, and more recently JSON, to help me better understand how our government works. I've worked hard to support open data and APIs in the federal government since 2012, resulting in me heading to Washington DC to work on open data projects at the Department of Veterans Affairs (VA) as a Presidential Innovation Fellow (PIF)

I understand how hard it is to do open data and APIs in government, and I am a big supporter of those in government who are working to open up anything. I also feel there is so much work left to be done to augment these efforts. While there are thousands of datasets now available via Data.gov, and in the handful of data.json files published by federal agencies, much of this data leaves a lot to be desired, when it comes to actually putting it to use.

As people who work with data know, it takes a lot of work to clean up, and normalize everything--there is just no way around this, and much of the government data that has been opened up, still needs this janitorial work, as well conversion into a common data format like JSON. When looking through government open data you are faced with spreadsheets, text files, PDFs, and any number of other obscure formats, which may meet the minimum requirements for open data, need a lot of work to get it truly ready for use in a website, visualization, or mobile application.

Adopta.Aency is meant to be an open blueprint, to help target valuable government open data, clean them up, and at a minimum, convert them to be available as JSON files. When possible, projects will also launch open APIs, but the minimum viable movement forward should be about cleaning and conversion to JSON. Each project begins with forking the Adopta Blueprint, which walks users through the targeting, cleaning, and publishing of data to make it more accessible, and usable by others.

Adopta.Agency employs Github repositories for managing the process, storage and sharing of data files, while also acting as gateway for accessing the APIs, and engaging in a conversation around how to improve upon data and APIs available as part of each project (which is what APIs are all about). Adopta is not a specific technology, it is a blueprint for using commonly available tools and services, to move government open data forward one or two steps. 

We feel strongly that making sure government open data available in a machine readable format, can be a catalyst for change. Ironically, even though this data and APIs are meant for other computers and applications, we need humans to step up, and be stewards of an ongoing portion of the journey. Government agencies do not have the skills, resources, and awareness to do it all, and when you actually think about the big picture, you realize it will take a team effort to make this happen.

Adopta.Agency is looking to define a Github enabled, open data and API fueled, but ultimately human driven network to help everyone realize the potential of open data and APIs in government -- please join us today.

from http://ift.tt/1KmmeAe

Thursday, August 13, 2015

Forget Uber, If You Build A Platform That Feeds People Like This, Then I Will Care

I was listening to the To Cut Food Waste, Spain's Solidarity Fridge Supplies Endless Leftovers segment on NPR today, which made me happy, but then quickly left me sad regarding 99% of the tech solutions I see being developed today. The tech sector loves to showcase how smart we all are, but in the grand scheme of things, we are mostly providing solutions to non-problems, when there is a world full of real problems needing solved.

I remember being at MIT for a hackathon a couple years back, where when we were done with the catered food for our event, the food was taken down to a corner in a hallway, that had a table, and a webcam. After putting the bagels, pizza, juice, and other items on the table, within about 20 minutes, it was gone--students fed, and food not wasted. #winning

The solidarity fridge idea reminded me of this experience, and it makes me sad that there is not an Uber for fucking feeding people! Why the hell isn't there a solidarity fridge and pantry on every street corner in the world? Why don't we have Uber for this idea? Why aren't there food trucks doing this? Oh, because there is no fortune to be made on actually making sure people are being fed, and Silicon Valley really doesn't give a shit about solving real problems, it is just what we tell ourselves so we can sleep at night.

If you are building a platform that helps neighborhoods manage their solidarity fridge and pantries, complete with API, mobile and web apps, and SMS push notifications, then you will see me get real excited about what you are doing--until then...

from http://ift.tt/1Ne4vjO

Wednesday, July 22, 2015

Micro Attempts At Being The Change I Want To See in Government

One by-product of being as OCD as I am, is that I am always looking for the smallest possible way that I can help grease the wheels of the API economy. A big part of helping the average person understand any company or API, is possessing a simple image to represent the concept, either a screenshot, logo, or other visualization. A picture is worth a thousand words, and as essential to API operations, as your actual service.

As I worked to understand the agencies that power our federal government, I quickly realized, I needed a logo for each of the 246 federal agencies--something that didn't exist. I could find many on Wikipedia, and Google for the others, but there was no single source of logos for federal agencies--even at the Federal Government Directory API from USA.gov. Unacceptable, I created my own, and published to Github. 

Ultimately, I am not happy with all of the logos I found, and think it can be greatly improved upon, but it provides me with a base title, description, and image for each of our federal agencies. It is something you can find in the Github repository for my federal government API research, and a JSON representation of all federal agencies + logos under the data folder for the research.

It took me about 6 hours to do this work, and it is something I know has been used by others, including within the federal government, as well as across numerous of my own API research, and storytelling. These are the little actions I enjoy inflicting, helping to wield APIs, and machine readable, meaningful, openly available, micro data-sets that can be used in as many scenarios as possible. Logos might seem irrelevant in the larger open data war, but when it comes to the smaller skirmishes a logo is an important tool in your toolbox.

from http://ift.tt/1SD5r0a

Friday, July 3, 2015

Use Of APIs By Regulators To Audit Telco Behavior

I keep reading stories about federal regulators investigating, and issuing fines to telcos like AT&T paying $105 million for unauthorized charges on customer bills, and Verizon and Sprint to pay $158 million for cramming charges on customers' bills. Maybe I am biased (I am), but I can't help think about the potential for APIs, and OAuth to help in this situation.

As an AT&T, and Verizon customer, I can say that I could use help in auditing my accounts. I'm sure other users would pay for a service that would help monitor their accounts, looking for irregularites. I think about services like Cloudability, that help me manage costs in my cloud computing environement--why aren't there more of these things in the consumer realm?

If all services that are available online simply had APIs for their accounts, this would be possible. It would also open up the door for government agencies, and public sector organizations to step up and provide research, auditing, and potentially data counseling for the average citizen and consumer. 

I want more access to the data I generate via the telcommunication companies. I also want to be able to take advantage of services that help me manage my relationships with these companies. I also think there should be a certain amount of regulatory acess and control introduced into all of this, and APIs provide not just a programmatic way to do this, it can be done in a real-time way, that might provide the balance we need--rather than just every few years when the feds have the information they need, and the enforcement power they need to take action.

from http://ift.tt/1H5wvCf

Friday, June 19, 2015

A Better Understanding When It Comes To Licensing Of Data Served Up Through APIs

Through my work on API Evangelist, and heavy reliance on Github, I have a pretty good handle on the licensing of code involved with APIs--I recommend following Githubs advice. Also derived from my work on the Oracle v Google copyright case, and the creation of API Commons, I have a solid handle on licensing of API interfaces. One area I am currently deficient, and is something that has long been on my todo list, is establishing a clear stance on how to license data served up via APIs.

My goal is to eventually craft a static page, that helps API providers, and consumers, better understand licensing for the entire stack, from database, to server, the API definition, all the way to the client. I rely on the Open Data Commons, for three licensing options for open data:

I am adding these three licensing options to my politics of APIs research, and will work to publish a single research project that provides guidance in not just licensing of data served up through APIs, but also addresses code, definitions, schemas, and more. 

The guidance from Open Data Commons is meant for data owners who are looking to license their data before making available via an API, if you are working with an existing dataset, makes sure and consult the data source on licensing restrictions--making sure to carry these forward as you do any additional work.

from http://ift.tt/1L7yrwO

Monday, May 11, 2015

On Encountering Skeptical Views Around Open Data

I spend a lot of time talkng about open data in business, and government of all shapes and sizes. This topic was front and center at APIDays Berlin / APIStrat Europe, and APIDays Mediterranea. Open data was a part of numerous talks, but most importantly dominated conversations in the hallways, and late into the night at the drinking establishments we gathered.

In my experience there are four camps of people when it comes to open data:

  1. Those who know nothing about open data
  2. Those who don't know much, but have lots of opinions
  3. Those who have experience, and over promise the results
  4. Those who have experience, and get hands dirty

I'd say overwhelmingly the people I met in my latest travels were in the first bucket, or the fourth bucket. However I did meet a handful of folks who I put in the second bucket, and were very dismissive of the potential of open data. In my experience these people either listened to the rhetoric of people in bucket three, or just don't have the experience that many of the rest of us have.

I agree that the state of open data our of city, state, and federal level government programs is often lacking much of what we'd like to see in a healthy, mature program. What I feel skeptics miss, is hands on experience making this happen in government (this shit is hard), and a willingness to help take things to the next level. This takes an effort from all of us, not just the people in government--there is a lot you can do from the outside to help make things better (not just criticize).

It feels like we are getting past a lot of the damage created by early open data rhetoric, that I felt over-promsied, and under-delivered. Something we have to learn from in future storytelling. I don't feel like all open data skeptics, and critics are required to get their hands dirty, but I guarantee if you work on a couple of hands on projects--your views will change.

from http://ift.tt/1dYmyMY

Tuesday, May 5, 2015

Shhhhh, Be Very Very Quiet, I Am Hunting Mainsplainers

For the most part I ignore the bullshit that flows in my girlfriend @audreywatters Twitter timeline (yes I am watching). We both tend to write some pretty critical things about technology, but for some reason (hmmm, what could it be), her timeline is full of some pretty vocal "dudes" looking to set her straight. I just do not have the energy to challenge every sexist male, looking to tell her she is wrong, but every once in a while I just need to vent a little--so I go hunting mansplainers in her Twitter timeline. 

One young white fellow, wins the prize this week, he got my attention, resulting in a conversation that ended in this response:

Yeah, the days she was writing that, and were discussed all the details, gave me no insight into the logic, let alone the last five years of discussing this topic with her. During my mainsplainer hunting, I'm not out to convince these dudes of how out of line they are, honestly I'm just looking to fuck with them, and let them know I'm here. I do not know the answer to helping us sexist men learn the error our ways. Yes, even I have sexist tendencies--only difference is that I am well on my way to learning. You see I am white, male, and even though I grew up very poor, raised by a single mother, I still have enjoyed a very priveleged existence for most of my life. 

I could easily cherry pick specific Tweets from this dude, showing his flipping flopping nature, where he blames Audrey for specific things he can't actually cite in her post, and talks of her blaming these other men he's defending for doing what he claims as sinister things, wait no sinister was his reference in Twitter conversation with someone else. No wait, the last paragraph in her post alludes to this. I just need to be able to follow the Twitter thread to understand his point. Why am I so dense?

Look, I don't give a shit buddy. I'm just fucking with you because you are spouting stupid shit in her timeline. I really don't give a fuck where you are coming from. If you knew the number of dudes I've seen tell her how wrong she is, to she needs to shut the fuck up, to hacking my websites and telling me to keep her in line, you'd go away pretty quickly (you are in good company). You need to tune into the bigger conversation, and not feel the need to tell women they are wrong. The reason you feel this way is you don't see her as expert because she is a woman. Period. 

What people like you should do is, write a response on your own blog, in your domain, and reply simply with "here are my thoughts". Then you can lay out all the detail you need, cite your own sources, and hopefully do as much work as she did when crafting her story. Then if she cares (she won't), she can reply on her blog, and return the favor to you. I know what you are going to say, oh I can't even open my mouth without mainsplaining? Probably not. You are clueless of the bigger picture, except the view from your own position.

I'm not saying everything that Audrey says is right, but I am saying you need to step back, and analyze your approach. One thing I've learned during my time running a business with my ex-wife, and the amazing five+ years I've spent with Audrey, is there is more to this, then us men can ever imagine. I disagree with a lot of things I read online, most of them I do not ever respond to, and the things I do, I critically evaluate how I respond--I just do not vomit my priveleged position into people's timeline. 

I know, futile effort. I can never change these types of people's behavior, but I just can't help hunting the mainsplainers in her timeline, and vent, while letting them know I'm sitting by her side. If you have any other comments or questions, please read Is My Girlfriend Bothering You?


from http://ift.tt/1QiriLg

Thursday, March 12, 2015

I Have Gotten More Return On the Ideas I Have Set Free Than Any I Have Locked up

When I walkthrough the junkyard of startups, and business ideas in my mind, I can’t help but feel that much of my current success with API Evangelist has more to do with open ideas, than it does any other aspect. I have numerous startups under my belt, where I tried to capitalize on ideas I've had, ranging from selling wine online to real estate data, but nothing like what I'm doing now.

Do not get me wrong, I’ve had a lot of success along the way, but nothing that compares to the feelings of success I have with API Evangelist. Other than right place, and at right time, I cannot come up with much that is different from API Evangelist, and previous work—except the fact that API Evangelist is focused on opening up and freeing every idea that comes along.

This type of approach to business might not be right for everyone. I’m sure I’ve also passed up on some pretty lucrative opportunities to monetize around my ideas, but in the end, I mostly enjoy making enough money to get by, and generating as much positive exhaust around my ideas as I can. I'm not saying all business needs to think like this, but the more API-centric your business is, I think the more you have to consider the repercussions of locking up ideas vs. setting them free.

from http://ift.tt/1wBIW70

Monday, February 23, 2015

Making Sense At The100K Level: Twitter, Github, And Google Groups

I try to make sense of which companies are doing interesting things in the API space, and the interesting technologies that are done by these companies, that sometimes take on a life of their own. The thing I wrestle with with constantly, is how do you actually do this? The best tools in my toolbox currently are Twitter and Github. These two platforms provide me with a wealth of information about what is going on within a company, or specific project, the surrounding community, and the relationships they have developed (or not), along the way.

Recently I’ve been spending time, diving deeper into the Swagger community, and two key sources of information are the @swaggerapi Twitter account, and the Swagger Github account, with its 25+ repositories. Using each of these platform APIs, I can pull followers, favorites, and general activity for the Swagger community. Then I come up against the SwaggerSocket Google Group. While there is a rich amount of information, and activity at the forum, with a lack of RSS or API, I can’t make sense of the conversation at a macro level, alongside the other signals I’m tracking on—grrrrrr.

At any time I can tune into the activity on Twitter, and Github for the Swagger community, but the Google Group takes much more work, and I have to go to the website to view, and manually engage. Ideally I could see Twitter, Github, and Google Group activity side by side, and make sense of the bigger picture. I can get email updates from the forum, but this applies from now forward, and gives me no context of history of the conversation within the group—without visiting the website.

Just a side rant from the day. This is not a critique of the Swagger community, just an outside view on the usage of Google Groups as an API community management tool. I use the platform for APIs.json and API Commons, but I think I might work on a better way to manage the community, one that allows outside entities to better track on the conversation. 

from http://ift.tt/1Eqi2jU

Sunday, February 8, 2015

Emails From People Saying Nice Things And Not Wanting Anything From Me

I process my thoughts through stories on my blogs, and often times you'll find me bitching about people and companies here on kinlane.com. Other times you'll find me waxing poetic about how nice people can be—welcome to my bipolar blogging world.

In this post, I want to say how much I like finding nice emails from people in my inbox, especially when they don’t want anything from me. Getting these nice notes from people, about specific stories, topics, or just generally thanking me for what I do, makes it all worth it.

Ok, I'll stop gushing, but I just wanted to say thank you—you know who you are.

from http://ift.tt/1A91Cee

Friday, February 6, 2015

An Archive.org For Email Newsletters Using Context.io

I’m not going to beat around the bush on this idea, it just needs to get done, and I just don’t have the time. We need an archive.org for email newsletters, and other POP related elements of the digital world we have created for ourselves. Whether we love or hate the inbox layer of our life, it plays a significant role in crafting our daily reality. Bottom line, we don’t always keep the history that happens, and we should be recording it all, so that we can pause, and re-evaluate at any point in the future.

I cannot keep up with the amount of newsletters flowing into my inbox, but I do need to be able to access this layer, as I have the bandwidth available to process. Using Context.io, I need you to create an index of popular email newsletter indexes that are emerging. I feel like we are seeing a renaissance in email, in the form of the business newsletter--something I don't always have the time to participate in.

During the course of my daily monitoring, I received an email from Congress.gov, about a new legislative email newsletter, something that seems like something I’d be interested in, but then immediately I’m questioning my ability to process the new information:

  • A specific bill in the current Congress - Receive an email when there are updates to a specific bill (new cosponsors, committee action, vote taken, etc.); emails are sent once a day if there has been a change in a particular bill’s status since the previous day.
  • A specific member’s legislative activity - Receive an email when a specific member introduces or cosponsors a bill; emails are sent once a day if a member has introduced or cosponsored a bill since the previous day.
  • Congressional Record - Receive an email as soon as a new issue of the Congressional Record is available on Congress.gov.

This is all information I’m willing to digest, but ultimately have to weight it alongside the rest of my information diet—a process that isn’t always equitable. If I could acknolwedge an email newsletter as something that I’m interested in, but only when I had time, I would be open to the adoption of a new service.

We need to record this layer of our history, something that our inboxes just aren’t doing well enough. I think we need a steward to step up, and be the curator of this important content that is being sent to our inboxes, and doesn’t always exist on the open Internet. Technically I do not think it would be too difficult to it using Context.io, I just think someone needs to spend a lot of time signing up for newsletters, and being creative in crafting the interface, and index for people to be able to engage with in meaningful ways, that people will actually find useful and pay for.

from http://ift.tt/1KoYwWh

Tuesday, February 3, 2015

A Machine Readable Version of The Presidents Fiscal Year 2016 Budget On Github

The release of the the president's fiscal year 2016 budget in a machine readable format on Github was one of the most important things to come out of Washington D.C. in a while when it comes to open data and APIs. I was optimistic when the president mandated that all federal agencies need to go machine readable by default, but the release of the annual budget in this way is an important sign that the White House is following its own open data rhetoric, and something every agency should emulate.

There is still a lot of work to be done to make sense of the federal budget, but having it published in a machine readable format on Github saves a lot of time, and energy in this process. As soon as I landed on the Github repository, clicked into the data folder, and saw the three CSV files, I got to work converting them to JSON format. Having the budget available in CSV is a huge step beyond the historic PDFs we’ve had to process in the past, to get at the budget numbers, but having it in JSON by default, would be even better.

What now? Well, I would like to make more sense of the budget, and to be able to slice and dice it in different ways, I’m going to need an API. Using a Swagger definition, I generated a simple server framework using Slim & PHP, with an endpoint for each file, budauth, outlays, and receipts. Now I just need to add some searching, filtering, paging, and other essential functionality, and it will be ready for public consumption--then I can get to work slicing and dicing the budget, and previous years budgets in different ways.

I already have my eye on a couple D3.js visualizations to help me make sense of the budget. First I want to be able to show the scope of budget for different areas of government, to help make the argument against bloat in areas like military. Second, I want to provide some sort of interactive tool that will help me express what my priorities are when it comes to the federal budget--something I've done in the past.

It makes me very happy to see the federal government budget expressed in a machine readable way on Github. Every city, county, state, and federal government agency should be publishing their budgets in this way. PDF is not longer acceptable, in 2015, the minimum bar for government budget is a CSV on Github—let’s all get to work!

from http://ift.tt/1DCnz46

Saturday, January 31, 2015

My Smart Little (Big) Brother And Programmatically making Sense Of PDFs

I was in Portland, Oregon a couple of weeks ago, and one of the things I do when I visit PDX is drink beer with my little (big) brother Michael (@m_thelander). He is a programmer in Portland, working diligently away at Rentrak. Unlike myself, Michael is a classically trained programmer, and someone you want as your employee. ;-) He’s a rock solid guy.

Anyhoo. Michael and I were drinking beer in downtownt Portland, and talking about a project he had worked on during an internal hackathon at Retrak. I won’t give away the details, as I didn’t ask him if I could write this. :-) The project involved the programmatic analysis of thousand of PDFs, so I asked him what tools he was using to work with PDFs?

He said they were stumbling on the differences between the formatting of each PDF, and couldn’t get consistent results, so they decided to just save each page as an image, and used the tesseract open source OCR engine to read each image. Doing this essentially flattened the differences between PDF types, giving him additional details provided when you use tesseract.

It may not seem like much, but ultimately it is a very interesting approach, and as I continue doing big data projects around things like patents, I’m always faced with the question—what do I do with a PDF? I will have to steal (borrow) from my smart little brothers work and build a tesseract API prototype.

from http://ift.tt/1yYL8Es

Wednesday, January 28, 2015

Why Are You So Hard To Get A Hold Of?

This is another post in my ongoing series of regular responses I give to people. Meaning when I get asked something so much, I craft blog posts that live on kinlane.com, and I reply to emails, tweets, etc. with a quick link to my standardized responses.

One I get less frequently, but still enough to warrant a response to, “why are you so hard to get a hold of?"

To which the answer is, "I’m not". I have a phone number that are very public, I have 3 emails all going into same inbox, a highly active Twitter, LinkedIn, Facebook, and Github presence. If you are having trouble getting a hold of me, it is because you are not using the right channels, or potentially the right frequency.

First, I don’t talk on the phone. I schedule meetings, increasingly only on Thursdays (regularly for partners, etc.), where i talk on skype, ghangout, and occasionally the phone. When I talk on these channels, I can do nothing else. I can’t multi-task. I am present. If I did this all the time, I wouldn’t be the API Evangelist—I’d be that phone talker guy.

Second, I respond well to quick, concise emails, tweets, wall posts, and github issues. Shorter, the more concise the better. This is what I mean by frequency, if you send me a long-winded email, there is good chance it could be weeks or even never that will respond. Sorry, I just don’t have the bandwidth for that frequency—I use short, precise signals.

I do not have a problem with someone being a “phone person”, but I’m not, sorry. In my experience people who require lots of phone calls, also require lots of meetings, and often shift in their needs, because it isn’t anchored to any specific outline, document, or project requirements. Personally I try to avoid these types of personalities, because they have proven some of the least efficient, and most demanding relationships in my professional life.

Please don't take this message the wrong way, I'm trying to help you be as successful as you can in making the right connection.

from http://ift.tt/1tvcNME

There Is A Good Chance That I Will Be Remembered For What You Did, Because I Told The Story

My friend Matthew Reinbold (@libel_vox) wrote a great piece on his blog titled, Storytelling and The Developer’s Need To Communicate, reflecting on an un-conference session I did last July at API-Craft in Detroit. Thanks for the great thoughts on storytelling Matt, something that is super infectious, and has reminded me a related story, which I hope continues to emphasize the importance of storytelling in API space.

Another one of my friends that I thoroughly enjoy swapping stories with at API conferences, and in the dark corners of bars around the world, is Mike Amundsen (@mamund). Now I may have the name wrong, but one time Mike told me a story about how John von Neumann (correct me if I’m wrong Mike), is known for a lot of ideas that he didn’t necessarily come up with on his own. He was just such a prolific thinker, and storyteller, which allowed him to process other people’s ideas, then publish a paper on the subject before anyone else could. Some people would see this as stealing of ideas, but one can also argue that he was just better at storytelling.

While I have developed many of my own ideas over the years, much of what I write about is extracted from what others are up to across the API space. I have made an entire career out of paying attention to what technologists are doing, and telling a (hopefully) compelling story about what I see happening, and how it fits into the bigger API picture. As a result, people often associate certain stories, topics, or concepts to me, when in reality I am just the messenger—something that will also play out in the larger history, told in coming years.

I’m not that old, but I’m old enough to understand how the layers of history lay down, and have spent a lot of time considering how to craft stories that don’t just get read, but they get retold, and have a way better chance of being included in the larger history. As Matthew Reinbold points out, all developers should consider the importance of storytelling in what they do. You don’t have to be a master storyteller, or super successful blogger, but your ideas will be much better formed if storytelling is part of your regular routine, and the chances you will be remembered for what you did, increases with each story that you tell.

from http://ift.tt/1CzDYaY

Tuesday, January 27, 2015

Cybersecurity, Bad Behavior, and The US Leading By Example

As I listened to the the State of the Union speech the other day, and stewed on the topic for a few days, I can’t help but see the future of our nations cybersecurity policy through the same lens as I view our historic foreign policy. In my opinion, we’ve spent many years behaving very badly around the world, resulting in very many people who do not like us.

Through our CIA, military, and general foreign policy we’ve generated much of the hatred towards the west that has resulted in terrorism even being a thing. Sure it would still exist even if we didn’t, but we’ve definitely fanned the flames until it has become the full-fledged, never-ending profitable war it has become. This same narrative will play out in the cybersecurity story.

For the foreseeable future, we will be indundated in stories of how badly behaved Russia, China, and other world actors are on the Internet, but it will be through our own bad behavior, that we will fan the flames of cyberwarfare, around the world. Ultimately I will be be reading every story of cybersecurity in the future, while also looking in the collective US mirror.

from http://ift.tt/1uZSQyK

Thursday, January 22, 2015

I Judge API Providers On How Much Value They Give Back vs. What They Extract

There are a number of data points I evaluate people and companies on while monitoring the API space, but if I had to distill my evaluation of companies down to one things, it would be based upon how much value they give back to the community vs. how much they extract.

You see some companies are really good about providing value to the community beyond just their products and services. This is done in many ways, including the open sourcing of tools, creation of valuable resources like white papers and videos, or just being active in sharing the story behind what they do.

Then there are companies who seem to be masters at extracting value from developers, and the wider API community, without ever really giving back. These companies tend to focus specifically on their products and services, and rarely share they code, knowledge, or other resources with the wider API space.

I’m not going to name specific examples of this in action, but after four years of operating in the space it is becoming easier to spot which camp a company exists in--you know who you are. I understand companies have to make money, but I’m totally judging companies across the API space based upon how much value they give the community vs how much they extract during their course of operation.

from http://ift.tt/1yLGdWc

Wednesday, January 21, 2015

When Will My Router Have Docker Containers By Default?

This is something I’m working on building manually, but when will the wireless router for my home or business have Docker container support by default? I want to be able to deploy applications, and APIs either publicly or privately right on my own doorway to the Internet.

This would take more work than just adding storage, compute, and Docker support at the router level. To enable this there would have to be changes at the network level, and is something I’m not sure telco and cable providers are willing to support. I’ll be researching this as a concept over the next couple months, so if you know of any read-to-go solutions, let me know.

It seems like enabling a local layer for docker deployment would make sense, and help move us towards a more programmable web, where notifications, messaging, storage, and other common elements of our digital lives can live locally. It seems like it would be a valuable aggregation point as the Internet of Thing heats up.

I could envision webhooks management, and other Internet of Things for household automation living in this local, containerized, layer of our online worlds. This is just a thought. I’ve done no research to flush this idea out, which is why its here on kinlane.com. If you know of anything feel free to send information my way.

from http://ift.tt/1E4bY16

Machine Readable Format For Migrating Concepts From Dreams Into The Real World

Obviously I’m working with APIs.json and Swagger a little too much, because it has really started to affect my dreams. Last night I had a dream where I was working with a university research team to define a machine readable format for migrating concepts from the dream world into the physical world.

I’m not sure I want this machine readable, but regardless it was a fun dream, and I wasn’t worried about this in the dream, so I guess it is ok. In the dream I was able to go to sleep and dream about a concept, then wake up and apply the same concept in my regular day via my iPhone. It allowed me to pick and choose from a notebook of things I had experienced in my dreams, and then apply in my daily life as I chose.

This post lives in the grey space between my fictional storytelling, and my API Evangelist storytelling, so I’ll leave it here on Kin Lane. If you are planning a startup in this area, let me know. ;-)

from http://ift.tt/15az6vw

Thursday, January 8, 2015

Internet Of Things Security And Privacy Will Always Begin With Asking If We Should Do This At All

As I read and listen to all of the Internet of Things stories coming out of CES, I’m happy to be hearing discussions around privacy and security, come out of the event. I feel better about IoT security and privacy when I hear things like this, but ultimately I am left with overwhelming concern about of the quantity of IoT devices.

There are many layers to securing IoT devices, and protecting the privacy of IoT users, but I can't help but the think that Internet of Things security and privacy will always begin by asking ourselves if we should be doing this at all. Do we need this object connected to the Internet? Are we truly benefiting from having this item enabled with cloud connectivity?

I'm going to try and keep up with tracking on the API layer being rolled out in support of IoT devices, but not sure I will be able to keep up with the number of devices, and the massive amount of hype around products and services. At some point I may have to tap out, and focus on specific aspects of IoT connectivity ,around what I consider the politics of APIs.

from http://ift.tt/1Dr71iv

Wednesday, January 7, 2015

Information Sharing And Collaboration In Government With The 18F Jekyll Hub Template

I’m a big fan of Jekyll based sites. All of the API Evangelist network runs as over 100+ little Jekyll sites, within Github repositories, via Github Pages. This is more than just website technology for me, this is my workspace. When you come across a half finished listing of contacts, or building blocks for a particular industry, or possibly a story that isn't fully edited—this is all because you are wandering through my API industry workshop. (pardon the dust)

Over the holidays, my girlfriend Audrey Watters (@audreywatters) has completed her migration of Hack Education and her personal blog Audrey Watters, to a Jekyll based mode of operation. Read her own thoughts about the new found freedom Jekyll is giving her over her content, data, workflow and the publishing of her projects—she is pretty excited.

Like APIs, a Jekyll approach to projects is way more than the technology. It is hard to articulate to folks the freedom, collaboration, flexibility, and transparency it has the potential to  introduce. It is something you have to experience, and see in action before you can fully understand, but I also have to ackknowledge that the transparency introduced by this way of working will not be for everyone.

I originally learned what I know about Jekyll from watching leaders in the federal government space, most specifically Development Seed, and round one Presidential Innovation Fellow, and now full-time Githubber Ben Balter (@BenBalter). Continuing this trend, it makes me happy to see 18F, out of the GSA, providing the 18F Hub, “a Jekyll-based documentation platform that aims to help development teams organize and easily share their information, and to enable easy exploration of the connections between team members, projects, and skill sets.” The 18F Hub is similar to the Developer Hub templates that 18F published, but I think holds a lot of potential in helping on-board a non-developer audience to the concepts of Jekyll,and  Github—hopefully making the social coding platform a little less intimating.

I do not think Jekyll and Github is for everyone. I’m not in the business of evangelizing one platform to rule them all, but I do think Jekyll itself, whether you run on Github, Amazon S3, Dropbox, or your own hosting or internal network environment, is a powerful tool for any project. I’m eager to keep an eye on what agencies put the 18F Jekyll templates to use, because it will signal for me that there are other healthy things going on at the agencies that do.

from http://ift.tt/14vZpNx

Tuesday, January 6, 2015

Playing Around With Jekyll Job APIs To Manage My Github Pages

I’m playing around with a concept right now that I’m calling "Jekyll jobs". As you may know, all of my websites use Jekyll, and run on Github Pages. Currently I have over 100 separate repos, and managing the content, and data across these repos can get complex.

I use a standardize approach I call “hacker storytelling” for publishing each of my projects, so I have a handful of things I need to update, ranging from the look of the site, to changes across all Jekyll posts, or pages. To help me keep things orderly and efficient I’m considering a lightweight, API driven, jobs framework to help me manage.

I am looking to make many of these “jobs” available to my girlfriend as well, allowing her to target specific resources, with specific jobs. Some of the jobs I’ve outlined are:

  • Link Management - Help me catalog, and manage the health of links that are used across all blog posts. A lot of links change, go bad, or any other numerous illnesses that occur.
  • Image Management - Help me catalog, optimize, and manage images that are used in my blog posts. I’m pretty good about manually doing a lot of this, but I sure could use help.
  • HTML Management - Sometimes HTML code gets ugly, either because I wrote it and didn’t give it the attention it needed, or possibly because it was generated out of another system, either way there is cleanup and maintenance from time to time.
  • Content Management - After I write a post I like to constantly re-evaluate tagging, indexing, and providing additional links to related content.
  • Content Indexing - My search across all of my Jekyll drive sites is not the best, and I’d like a way I can index all, or specific collections, and serve up as simple search API, maybe using ElasticSearch or something.

As more of my world runs as small, modular, Jekyll projects, I’m needing a way to run jobs against them, and designing APIs that do what I need, and use the Github API to work with my Jekyll site content, makes sense. I’m thinking I will just pass a Github user, and repo name, as parameters to each Github job API, and have it run a specific task against my _posts folder in the Jekyll install.

Since I’m designing these Jekyll jobs as APIs, I can run each one as an API request, and keep the job logic separate from each project. I’ll get a couple of these setup, than blog more about the pros and cons of this approach-who knows it may not be what I need to get it done.

from http://ift.tt/1FkY5x0