Quantitative productivity Quantifying Productivity

I ' m always on a lookout for interesting datasets to collect, analyze and interpret. And what better datasets to Collect/analyze than the meta-dataset of my own activity collecting/analyzing other DA Tasets? How much time does I *really spend working per day? How does I spend most of this time? What makes me productive? These is all relatively important questions which I ' d like answers to, and since I prefer my answers based on data and not Confirmation-bias-susceptible personal anecdotes, I wrote Ulogme.

"I prefer my answers based on data, not confirmation-bias-susceptible personal anecdotes"

I ' ve now collected my computer usage data over a period of almost 3 months. In this post I ' ll highlight some of the features of the project, some of the insights I were able to derive so far and some Thoughts about where I hope I can take it next. And who knows, maybe by the end of the post you'll want to become a user yourself:)

What ' s out there already

The idea of tracking and visualizing your computer activity are not at all new. It has been around in various shapes and forms in quantified self circles and several programs already exist this try to F Ill this need. Among the few better known ones is RescueTime and TOGGL, but there is literally tens to hundreds of other quite terrible Copies. Among all of these, I couldn ' t find anything that satisfies a few very simple, basic requirements:

• The user interface must be web-based because it ' s 2014
• Everything must is open source and free
• The data must never leave the local machine (No Cloud Mambo Jambo-too personal!)
• It must is easily customizable and look pretty

Nothing like this (by far, actually) exists, so I set out to implement my own solution.

Brief Tour of Ulogme:single Day View

Ulogme is small and simple:there are both backend components:a tracking script that records activity and a small Local Web server wrapper that serves the activity logs to the frontend (visualization pages). The tracking script currently records active window titles (at frequency of once every 2 seconds) and keystroke typing fre Quency.

Lets go through a brief overview of some of the resulting visualizations and features. First there is the single day view. Lets look at my August 1st, for example. The header tells us the day of the recording and there are space for a short "blog" Post which can be written up for each DA Y:

Header:day Information, refresh button, buttons for going between days, and a little editable ' blog ' Post for the day.

Now we start to get to the meat. It looks like I am in the office from 10AM to 8PM on this day. Now, the remember that we record keystrokes and window titles throughout. What follows are the keystroke breakdown for the day:

Keystroke statistics for the day.

We see this I spent most of the day coding in Sublime Text 2 (which I use to write python/js/c++) and gmail-looks like I Wrote quite a bit of email! Next, Ulogme shows the barcodeof the day, as I like to call it. This is a breakdown of the "the" The windows on?

Barcode of the day. Mousing over any of these strips reveals the exact window title.

This view was a little dense so let me unpack it one by one:

• The Notes feature (on top) allows me to enter arbitrary Notes for any time. Notice I also wrote an (optional) feature the looks for notes about coffee and calculates my levels of caffeine based on actualhalf-life of coffee. I am Curious what caffeine does to my productivity!
• I Group my Windows into  display groups   Barcodes , where the first group involves F Un (gmail/chrome/non-coding files opened in Sublime Text 2-such as  . markdown  for blogging) and the SE Cond the group involves work (Matlab/ipython Notebook with.js/.css/.cpp/.h/.py files, or PDF files opened (papers)). Looks like I spent roughly half of the day on work.
• Hacking Streak is a nifty feature the tries to identify contiguous Hacking activity and correlates reasonably WI th my productivity. It looks for active windows the constitute work (I define the settings) and then for continuous keystrokes above some Typing frequency threshold. This indicates that I ' m in a state of hacking, and the streak gets gradually interrupted if I switch windows to n On-working titles, or if I stop writing code. The longest one visible here is minutes and when I hover over the active title at that time, I see that it is me Addi Ng a feature to Ulogme. The longest I ' ve seen anyone get is a lab mate Beta tester friend with an intense 50-minute hacking streak.

In the end, Ulogme shows the final breakdown of titles that occupied me on this day:

The final breakdown of active window titles.

That's interesting, it looks like I actually only spent 10% of my day in Gmail. So even though I wrote a lot, it is just a few emails and chats I quickly sent out.

Brief Tour of Ulogme:global overview

Insights for one day was interesting, but everything becomes signficiantly more meaningful when it was put in context of a Large number of days. Perhaps you noticed the ' overview ' link on the header; Clicking this takes your to the overview page of Ulogme that takes the statistics for all days and puts them together. I recorded my activity for almost 3 months now. Here are the delicious data visualized for the entire period (with some overlayed annotations):

Total amount of time per day spent in various applications over a period of three months. The titles on top is clickable and toggle on/off the visualization of any one of the titles.

So AWESOME. There is many fun things to note:

• Note the deadline mode right before NIPS paper deadline on June 6th. I was frantically writing Latex for the most part:)
• Right after the deadline, you see a dip in activity. This is because I am mostly on my laptop preparing things for the CVPR conference where I had to give a talk. This points to one issue with Ulogme -There are no syncing across machines right now.
• Notice a few dips on Sundays-apparently Sundays is my rest days:)
• Am I Just hallucinating this, or was there a fairly significant jump in activity right after breaks (note very high bars RI Ght after CVPR and vacation.) This needs more data but it would is interesting if vacations actually made me more productive. We ' d had to measure more than just time spent on computer, though.
• When I toggle off all non-working titles, the Visualizaiton (not shown) reveals that I only spend somewhat depressingly Li Ttle time actually working. Many days I come into labs in the morning and leave late at night to go straight to sleep, but even these days sometimes ad D up to only roughly 5-6 hours of actual coding. I was very surprised about this initially and went looking for bugs, but it's true upon closer Inspection-there is a sh ORT commute, lunch, dinner, random reading groups, meetings, random slacking off on the internet, Gmail, etc etc ... It all builds up quite quickly! Depressing to see that quantified.

Next, Ulogme gives me nice breakdown for both keystrokes and time spent in every window, across all time:

Summary of keys and time per window across all 3 months.

This is a little incomplete because I does some hacking on my laptop and it paints an interesting picture nonetheless. It looks like I spent a good chunk of desktop time in Matlab, but seemingly I spend the most amount of time in Chrome Scre Wing around and browsing the Internet. Great.

What is it takes to write a paper. Note that, interestingly, my total time for Latex are hours-this is the how long it takes to write a paper! Additionally, I pressed 225,149 keys in my Latex editor and the "on $ wc -l my paper .tex file reveals" it has " c5>40,192 characters. Some of it is the template code but in least approximately, this means the it takes about 5.6 characters for every One character in the final paper!

It takes hours and 225,149 keys to write a 40,192-character NIPS paper (i.e. 5.6 characters must is typed for every one Final character.)

The final visualization is too long to paste this entirely, but I'll show a snippet:

Keystroke frequencies visualized for every day, along with the marginal sums on top and right.

This visualization seems to suggest that I did most of my work between 10AM and 8PM, and a very productive day are about 50, Keystrokes. You can also see a bit of my post-nips refactory period with much lower keystroke activity.

In the end, Ulogme tells me, 3 months I's ve pressed a total of 1,608,943 keys over Ays, or approximately 19,384 per day.

Going forward

Going forward, I ' m hoping to make Ulogme in a nice, open-sourced pet project. The code is all available Ongithub under MIT License and anyone are welcome try it out (if you ' re on Ubuntu or OSX -Windows is not supported, and if you ' re using a modern browser).

And if you ' re feeling extra adventurous, I warmly welcome pulling requests for new features or bug fixes. The code base is a mix of Python, Javascript and I use d3.js for all visualizations. The project is in fairly early stages and the code are not among the nicest I ' ve produced, but I ' ve started fairly major re Factoring efforts to make the onboarding process easier.

In longer term, I ' m hoping that Ulogme codebase would evolve to become beautifully modular set of Data View PLUGINS that could is customized, stacked up and composed in the user interface as desired.

Im Summary, I feel I ' ve gained quite a few insights into my own work habits by just visualizating the data, but there is M Uch more work-to-be-on-the-side as well. The Holy Grail is still isn't implemented:what is the correlated of my productivity? Does sleeping more help? Does drinking coffee help? Do vacations or breaks to all? All of these questions has answers and I can ' t wait to find them, in the data.

from:http://karpathy.github.io/2014/08/03/quantifying-productivity/

Quantitative productivity Quantifying Productivity