The Fundamentals of Calendar Hacking

I am always amused by time-management amateurs who have found a system that works for them and a few of their friends, and start imagining that they’ve created a perfect system.  “Universal time management system” is the perpetual motion machine of the self-improvement industry.

The zeroth thing you need to know about personal time management is that in a certain theoretical sense, there are no universal  systems. Only calendar hacks. What’s more, you cannot pick some compendium of calendar hacks and easily sort out the ones that will work for you. You need to learn the art of calendar hacking.

That’s what this post is about: the fundamentals of calendar hacking. I’ll be straight with you: the ideas in this post are going to be somewhat tough to grasp if you haven’t already encountered them, but I’ll keep it non-technical and provide several hopefully illuminating examples along the way.

The key is diagrams like the one below.

 

Diagrams like this are known as empirical computational complexity phase transition diagrams in computer science. I’ll show you how to read and draw informal, non-technical versions in a minute, but the key idea behind them is that an impossibly hard scheduling problem is not impossibly hard everywhere and at all times.

The key to calendar hacking is separating out the hard and easy regimes and dealing with them differently. This is one of my favorite technical ideas, and my excuse for playing fast and loose with it, as I am about to, is that my heart is in the right place. I mentioned this idea in a footnote somewhere in Tempo, but I figured I ought to do a proper post on the idea.

Defining “Impossibly Hard”

The “impossibly hard” bit (which is why there are no universal systems) is a worst-case feature rather than a typical-case feature. And by impossibly hard I mean you wouldn’t be able to solve them even if you had all of Google’s ninjas hammering away at the solution for you on a 1,000,000-server farm.

That’s actually a pretty good metaphor. If somebody built a “what to do next” engine that looked like a search engine, you’d get an answer back pretty quickly most of the time, but for a subset, the engine would just hang instead of delivering an answer in time. The calendering engine would be of no use during your worst weeks.

Imagine if regular Google search behaved that way. Search (at least in the form Google has defined and offered it) is not impossibly hard in this sense. You always get some answer.

But there is good news. It turns out, you can usually isolate these worst cases by partitioning the space of instances of the problem in the right way.

Once you master this divide-and-conquer principle, calendar management will remain a hard problem (assuming you’re up to something worthwhile with your life). But at least you won’t waste time doing futile things, and you’ll be able to pick your battles as you hack away at your personal workflow. Let’s see how.

How Good Can it Get?

The first order of business is to get a realistic sense of how good your calendar management can get in your life. Some lives are just more hellish than others.

To do that we need a rough definition of a calendar-management problem. Here’s mine.

For an individual, a calendar management problem is the problem of deciding if and when to do each of an unending, somewhat unpredictable stream of varied work and opportunities coming at you, with various criteria determining the actual or potential value of each item in the stream. The work and opportunities may arise from your own ideas or from demands other people make on your time. It doesn’t matter for our purposes.

Usually, the value of doing something varies with when you decide to do it, making things even more complex. And there are usually hard and soft constraints besides the presence of pre-existing tasks on the calendar, that determine whether you can do something at a given time. For example, just because you are idle between flights at an airport does not mean you can finish your grocery shopping.

Here are three things you should know about how good it can get in general:

• Some calendar management problems are impossibly hard only if you want an optimal solution that makes the best possible use of your time in some appropriate sense.
• But you’re not out of the woods even if you are willing to give up on optimality and live with “good enough.” Some scheduling problems are impossibly hard even at the “good enough” approximation level.
• And some are near impossible even if you want to find any solution at all, even a terrible one.  Yes, sometimes your calendar can kill you.

Unfortunately, without formally modeling your life in mathematical terms, defining a formal version of your specific calendar management problem, and hiring a computer science PhD to do the formal analysis, there is no easy way to tell how good your life can get. But a good default answer is “not very.” Most realistic calendar management problems for working adults are going to be impossibly hard under worst-case conditions.

Fortunately, you can usually get a good sense of how good it can get by looking at people with similar life-and-work styles.

So it is useful to keep a variety of different examples in mind, when thinking about calendar management. I call these calendar archetypes. Let’s take a quick tour of a few before talking about how calendar hacking.

Some Calendar Archetypes

The definition of calendar management problems reveals the enormous range possible. A GP at a busy American medical practice has a much simpler calendar management problem than a venture capitalist, who in turn has an easier time than a busy CEO.

For a GP, both the difficulty and financial value of each case is confined within a relatively narrow range, since tough cases get kicked up to specialists and the fees for each visit are standardized by HMOs in America. It is enough to schedule based on severity of condition and a first-come-first-serve secondary principle. Furthermore, clinic hours can be easily partitioned from the rest of the schedule. Due to the need for specialized diagnostic equipment, regulations and social norms, people don’t generally ask doctors for complicated medical advice at random times. The actual work of being a GP may be hard, but the calendar management is not.

For a VC on the other hand, the potential value of a deal can vary hugely, and the difficulty of assessing a given deal can also vary hugely from no-brainer fantastic deals to valuable but complicated ones requiring tricky due diligence. It is also harder to separate obvious deal-flow work (entrepreneurs asking for meetings) from serendipitous opportunities that arise at unexpected times, or opportunities that need to actually be created (for example, by doing some backstage match-making).

A CEO has an even tougher problem, since everything is deeply coupled, and no single, simple mental model like “portfolio management” will serve to frame the problem of managing the calendar.

In case you think only people who make pots of money have difficult calendar management problems, it doesn’t get any easier on the down-and-out end of the spectrum. The difference is that for the latter, the problems are often not worth solving, financially speaking. Nobody has an incentive to care.

Consider a student trying to pick the right classes to maximize bang for the buck in the college experience, with a view to maximizing lifetime returns, a problem that is surprisingly similar to the problem of a VC picking deals to invest in. Historically, this problem has not been worth solving properly. These days, it is. There  is an increasingly valuable product waiting to be built here.

Or consider a homeless person in a new city, faced with the problem of deciding where to panhandle and when, taking into account foot and car traffic at different times of the day at different intersections, presence/absence of cops, local laws, the role of the local underworld in running life on the streets, and the generosity of locals.

This calendar management problem is surprisingly similar to the CEO problem. There is a great Hindi movie called Gardish that has a comedic subplot involving a Bombay beggar rising from a struggling solo operation to CEO running a city-wide organization of beggars. This problem probably will not be worth solving in the foreseeable future, even for a nonprofit, unless George Clooney takes an interest.

Some calendar management problems involve enough financial value that smart people will swarm them and make progress. Others don’t, and the people affected by those problems must simply suffer.

Phase Diagrams as Default-Switching Curves

A note to the cognoscenti here: suspend your craving for rigor for a bit. I will appropriately qualify all this hand-wavy stuff at the end.

When you take a problem like a calendar management system for a given broadly defined situation such as “CEO life,” you are faced with a bundle of instances.

For a GP, a natural unit of analysis is a patient visit. For a CEO, you need a periodic analysis session simply to define the units of analysis. It might be “meetings” or “trips” for instance.

But once you have a space of instances, you can view your life as a stream of such instances coming at you, that need to be handled. Sometimes the stream has one character, and at other times it has a different character.

“Character” is of course a loose, qualitative notion. For a calendar, you might use words and phrases like empty, full, messy, chaotic, packed, wide open, grueling, on-the-road, humming along and running smoothly to describe the character.

The basic problem in hacking a calendar is figuring out the single most important measurable variable that captures a key watershed distinction in the space of “characters” that your calendar can assume.

So the first thing you need to do is find some sort of “knob” in the space of instances, such that when you turn it from minimum to maximum, you get a class of easy problems, a narrow class of worst-case hard problems, and another, different class of easy problems.  So your work stream will always be in one of the two easy phases or the hard phase.

Then you figure out the likely default answers for the first and third phases. This is something like designing a triage system, but not quite. It’s more like designing two parallel triage systems.

The phase diagram is basically a switching curve from one default answer to another, when faced with a particular instance. That’s it. You figure out whether you are in one of the easy zones and follow a specified default calendar-management behavior if you are, and put in more computing effort if you are not.

You can loosely interpret the black line as the probability that the correct answer is one of the defaults and not the other, and the red line as the computing difficulty in coming to an answer that is likely to be right. Obviously if the probability is high that one of the defaults is right, the computing effort is low: you can just use the default (or some simple heuristic) and live with the low probability that you were wrong.

Typical Phase Diagrams

The specific tuning knob depends on the problem, but a good general candidate for calendar management problems is subscription level, the ratio of demands on your time to time available. This is the one illustrated in the first diagram.

As your calendar gets increasingly packed, you go from a default yes answer to new opportunities/work to a default no answer.

This probability of yes is almost 100% in the left half, and almost 0% in the right half, so you won’t go wrong very often by simply saying yes/no by default in those zones, respectively. Which means you just need to know which side you are on, and that you are reasonably far away from the danger zone.

When you have a set of work items and opportunities, if there’s nothing at all in the calendar, it’s easy enough to prioritize things in some meaningful order (like “most valuable first”).  Something usually beats nothing.

Equally, when the calendar is nearly completely packed and you are heavily oversubscribed, it is easy to discard most waiting tasks and opportunities, because they won’t fit in any available hole, and are unlikely to offer a higher value than something that’s already on your calendar.

The hard part is the transition zone, when your schedule is somewhere between X and Y% packed. The crucial insight is that this range tends to be relatively narrow. For your particular situation, the hard cases may lie between 130-140% oversubscription.  Above that, the answer is nearly always “can’t do it” and below that, it is nearly always, “yeah, I can take that on.”

In between, life is fiendishly hard.

Non-Typical Phase Diagrams

Subscription level is a good default tuning knob because most of us are neither permanently under-subscribed or over-subscribed, though we may go through temporary phases in each state. It is a variable that naturally wanders over its full range.

But when you are dominantly under or over-subscribed, you typically have to look for other variables. Be careful not to look only at obvious “scheduling” type variables involving time, money or energy in some form.

The tuning knob for a particular CEO’s schedule might be the volatility of the company’s stock for instance.

Meaningful diagnostic questions for saying yes/no to demands on calendar time might be “does it help manage the company?” when volatility is low, and “does it help manage stock market expectations?” when it is high.

I am totally making this up, since I’ve never managed a public company, but not out of thin air. This particular pair of defaults is suggested by the Jack Welch quote that “anyone can manage for the short term, and anyone can manage for the long term, the challenge is managing both at once.”

If you equate the two to managing stock price and managing the company (which almost by definition is managing it with a view to long-term health fundamentals), you get the following hypothetical phase diagram, which I suspect is true for a lot of Fortune 100 CEOs. On the left you prioritize company management stuff. On the right, you prioritize stock-price management stuff.

 

 

Simple, Wrong Answers to Oversubscription

A particularly valuable feature of the phase diagram approach is that it helps you avoid a common failure pattern that oversubscribed people are prone to.

They are so flattered by the demands on their time that they make up calendaring heuristics that fill up time in pleasant rather than effective ways.

1. A CEO might pack his/her calendar with meetings with top reports, big clients and other “important” people, and convince himself/herself that they are spending their time in the most valuable way possible. They might actually be filling up time in ways that make them feel “important.”
2. An in-demand speaker/writer/consultant may say yes to every invited talk/seminar, enjoying the adulation and fat speaking fees, studiously ignoring the possibility that the fad driving demand might pass next year.
3. A VC with a big pile of money to give away might stop seriously analyzing people and markets with care and begin to rely entirely on pitching rituals and performances to make decisions, an American Idol model.
4. A President might shy away from tough legislative battles and spend all his time managing popularity ratings and trying to outmaneuver opponents in Congress around easy battles, to earn political points.
5. A surgeon might pack the schedule with high-success-rate surgeries.

The shared pathology here might be called “resting on laurels.” A successful person (hence the oversubscription) simply milks the success by taking on only easy challenges.

There is no shortage of purely arbitrary prioritization principles to help you rest on your laurels.

The value of the phase-diagram approach is that it forces you to be sensitive to at least two different easy modes of operation separated by a tough mode. It forces you to extract at least one bit of actual intelligence from your surroundings.

If your system of calendar management has at least two different gears, and you encounter the occasional period of extreme chaos (the “hard” regime), you pass at least a basic test of thoughtfulness.

I already gave you one candidate tuning knob for the CEO case, but you might want to think about good knobs for the other four cases.

The “Hard” Zone

The hard zone, once you’ve identified it, cannot be handled in automated ways. You may not get optimality, “good enough” (what computer scientists call “satisficing”) or even feasibility when you are in that zone.

The best you can do is muddle along, sigh that it is “one of those weeks,” look for ways to just survive, and hope for the best.

Here’s a pointer though: if you find that you are always in the hard zone, you’re in deep trouble. 

The hard zone being hard means that you will make mistakes while you’re in it. In fact, in the heart of the hard zone, you are going to do no better than coin tossing. The calendaring part of your job can be outsourced to a dime.

Generally what makes life constantly hard is not that the individual instances of calendaring decisions are so hard in an absolute sense, but that there are simply so many of them, coming at you non-stop.  This means they must be solved faster. Any problem can be made impossible if you demand that it be solved faster and faster.

Occasionally you should make the super-human effort and try to do better than random, but in general, you should rearrange your life situation so that you get a full range of play in your calendaring. Simplify things so that you’ve got a full phase diagram.

In fact, if you don’t mind getting absurdly meta, you can even think in terms of a phase diagram of tuning knobs, with the tempo of your life as the tuning knob. If the tempo of calendaring decisions coming at you is sufficiently sedate, subscription levels will nearly always work as a tuning knob. If you speed up any life sufficiently, nothing will work, and you’ll do no better than a coin in deciding what to do/not do.

 

 

A Concluding Example

Take my own situation. I find Paul Graham’s Maker Schedule/Manager Schedule a useful guide in managing my time. The defaults are to manage your time in 4-hour blocks (Maker time) or 1-hour blocks (Manager time), and deal with others differently whether they are themselves in Maker or Manager mode.

In my own life, I’ve had periods when I’ve been largely a Maker, and periods when I’ve been largely a Manager. Those were fairly easy to deal with. It was also easy to deal with periods when I had both types of activity going on in my life, but they were separated by infrequent switches. When I used to work a day job at Xerox, my work-day was mostly Manager time, while my personal life was mostly Maker time (writing).

Now, as a free agent, the two are getting mixed up in very muddy ways. My calendar was in an “easy” Maker-dominant zone in the past year, when my consulting load was relatively low, allowing me to easily separate writing and consulting time, and within both, Maker and Manager types of work.

Now that I am getting busier, I am heading towards the “hard” zone. I am not so busy or rich that I can afford to just say “No” to new interesting gigs, but on the other hand, my life is complicated enough that saying “Yes” or “No” is much harder. It’s not just a case of deciding whether the project interests me. I also have to figure out whether I can fit it into my life without everything going to hell.

I still try to do Maker/Manager work separation. My most basic time management decision these days is asking, thrice a day (when I get up, after lunch, and in the evening) whether I am going to try and Make something for the next 4 hours, Manage a bunch of things for the next 4 hours, or simply slack off and let myself dissipate energy by idling away on Facebook, Quora or reading randomly. The last category is important, it represents relaxation, social time and general situation awareness upkeep.

For me currently, subscription level still works as my tuning knob, but I am getting just oversubscribed enough that I need a different knob. I am currently struggling to figure it out.

I’ll stop here, as far as the main post goes. If you’ve already discovered this principle in your own life, I’d be curious to hear about tuning knobs that have worked for you.

Note to Technical Types

You can completely ignore this section if you are not familiar with computational complexity theory. If you are, you’re probably a bit mad about how I’ve mangled rigorous ideas.

If you some familiarity with recent research, you will have recognized the ideas in this post as being loosely derived from the work that started around 1991 with Where the Really Hard Problems Are by Cheeseman, Kanfesky and Taylor and Finding Hard Instances of the Satisfiability Problem by Cook and Mitchell. I kept up with the literature until about 2005.  There has been a flood of work in the two decades since the original findings, and phase diagrams have been developed for many NP-Complete/NP-Hard problems that map to everyday scheduling problems, including k-SAT, Hamilton Circuit, Traveling Salesman and so forth.

To my knowledge, nobody has figured out a theoretical model around this stuff or systematic ways to parametrize problem spaces and discover phase transition boundaries. I expect the latest editions of classic complexity texts probably have a more digestible technical treatment of the subject of empirical complexity. If there has been serious progress, somebody please educate me.

Three caveats are needed for the kind of loose application I’ve outlined here.

The first is that many apparently thorny problems, when you actually analyze them, aren’t NP at all, and the phase diagram model is moot. You can simply develop good polynomial time algorithms to solve even the worst cases.  My justification for making the leap of faith that typical personal planning and scheduling problems are NP is that most scholars in the field seem to do the same (in fact there was a remark along these lines in a paper that I cannot recall; I think it was a Dean/Kambhapati paper). The real-time constraint is key here.

The second caveat is that the problem space parametrization that somebody might come up with through an informal qualitative analysis might turn out to be not be the right one at all, upon further analysis. The leap of faith I make here is that for a sufficiently large number of people evolving a set of hacks through imitation and tweaking, the process can probably be relied upon to pop out good “folk” candidates (such as stock price volatility for a CEO’s life).

And finally, the third caveat is that an informal analysis, even if it somehow uncovers the right problem space parameterization, will certainly not yield usable bounds for the phase transition zone. Actually figuring out (say) oversubscription levels that mark the threshold is likely to be a seriously hard problem.  But for informal use, I don’t think this is a problem. Chances are, with practice applying a particular tuning-knob heuristic, the decision-maker will develop good intuitions for when he/she is near or far away from the phase transition zone.

With those caveats, I think this informal DIY empirical complexity analysis model is a fairly safe tool even in the hands of people without the appropriate technical background.

If not, oh well. Sue me.