Steve Grossi at Work

The Best Leadership Advice I’ve Ever Been Given

Fri, 08 Mar 2019 00:00:00 +0000

It was a spring afternoon a few years ago. After a walk around the neighborhood that Lessonly calls home, Max Yoder and I sat down and he told me something that I’ve remembered many times since then, advice that has enriched my relationships in life and work.

He told me I’d built up a lot of goodwill at work, that people appreciated and even trusted me, but that I wasn’t using that. At first I didn’t understand: shouldn’t helping people be an end in itself? Of course, but the goodwill you build up from doing good work and helping people can be used to do even better work and to help even more people. Change is hard, but if you believe in a change for the better, you can draw on some of the goodwill you’ve built up to ask others to try something different, to take a chance. Or maybe a teammate is doing something that’s not helping your team, even hurting it. Someone needs to tell them that, and if you’ve shown that person that you care about them and the team, you can be the one to speak up.

Max gave me the metaphor of a battery that recharges over time: if your goodwill battery is always full, you don’t get the benefit of its replenishment. Spend some of that charge when it matters, and trust that it’ll come back. I’ve always been a bit of a people-pleaser, so I like the feeling of my goodwill battery always being at 100% with everyone in my life. But I’m grateful to Max for pointing out the missed opportunities in doing that. He might have thought he was draining a bit of my goodwill battery in pointing that out, but I’ll say this: its capacity grew as a result.

Over the years, this advice has given me courage when I’ve needed it. As a manager on my team, the easiest thing in the world is to focus on making my team happy, and I do. But I sometimes need to have difficult conversations (difficult for me, at least) such as telling colleagues when their work could be better. Remembering that I’ve built up trust and shown I care helps me have those conversations without fear that the person I’m talking to will suddenly hate me. Outside of work, this advice has given me strength in conversations with family, with whom I don’t always agree politically. These are heavy times, and there are crucial conversations that need to happen about the climate, race, justice, and so much more. Those conversations won’t happen on a large scale if they don’t first happen on a small one. Now, I’ve always been “a good boy” and am unused to taxing my family’s patience, but remembering all the trust I’ve built up with them over the course of my life has given me the strength to share my own convictions, even and especially when they diverge from from the beliefs of those I love. And we’ve had some really meaningful conversations as a result.

This advice—to balance making people happy with challenging them to help you make things better—may be obvious to some, but a few years ago it was just what I needed to hear. And so, I want to share it. I hope you find it useful.

Retro Roulette

Sat, 05 Jan 2019 00:00:00 +0000

Frequent reflection on how we can do better work in better ways is one of my favorite things about my team’s culture. For a long time, this took the form of monthly “retro” meetings where we all get in a room for an hour and talk about what’s been fun or frustrating, and how we can do more of the former and less of the latter. However, as my team grew to around 10 people, we noticed a creeping resistance to speaking up which led to conversation feeling more artificial. This only got worse as we grew past 20, when we finally put the meeting out of its misery. I really missed these opportunities for reflection, and want to share a new approach we’ve been trying to encourage this kind of reflection on larger teams, with which we’ve seen promising results. I call it Retro Roulette.

The motivation for this was simple: if one big conversation wasn’t working, let’s try a bunch of smaller ones, but inject a bit of randomness so that folks still get to hear from colleagues they don’t work with every day.

The Method

It works like this:

Once a month, we put the names of everyone interested into a hat and draw pairs. (Not a real hat, I’ve modeled this in Ruby with names.shuffle.in_groups_of(2))
Each pair gets coffee, lunch, or just chats that day about the usual retro topics, or whatever’s on their mind.
If topics or ideas come up that someone wants to share or discuss with the whole team, we have a Slack channel for that.

It’s worked out pretty well so far: participation is optional, but most folks who joined at first continue to do so. (I’m still looking to reach those who haven’t given it a try.) There’s usually a post or two in our chat channel afterwards about something someone found interesting. And for me personally, it’s been a great opportunity to engage with teammates I don’t work with regularly.

Some Slight Modifications

Since its inception, we’ve made a few changes to Retro Roulette:

After starting with pairs, we’ve changed to conversation groups of three people, which the team almost universally preferred. There is research to suggest the ideal size of a conversation is around 4-5 people, so we’ll likely try at least one more increase.

And after starting without much direction to the topics of conversation, we’ve since tried suggesting topics, often pulling from resonant topics that came up by chance among earlier groups. Some of my favorites have been:

What’s something you want to try—in work or outside—in 2019?
Where do you want to be next year? In 3 years? How can your colleagues help you get there?
What’s the one thing we as a team could do differently, that would make everything else easier? (Inspired by the book The One Thing)
What’s a product or product team you admire? What do you admire about them? What would that look like here?

If you try (or have tried) something like this on your team, I’d love to hear about it.

The Many Motivations of a Programmer

Thu, 02 Aug 2018 00:00:00 +0000

A conversation with a friend yesterday prompted me to reflect on all the different ways I’ve felt motivated in my career, how motivations have come and gone, and how often they’ve changed not just between but within jobs.

At the start of my career in software, my primary motivation was to prove myself. As an internet-taught hacker (and not in the cool sense), I had no credentials or schooling to lend me confidence, so I labored mightily against imposter syndrome. This was a powerful motivator, but also a dangerous one. Balance between work and life went out the window, and I spent significant portions of my free time reading about or practicing programming to show my colleagues I could work at their level. It also caused me to take every “best practice” I learned much too far. “Don’t repeat yourself?” Here ya go: every string in this app is now a global constant so no string is ever repeated.

Ten years into my career, the imposter syndrome has gotten better but never goes away. I find it’s relative. I may be something of an expert on my team, but send me to a conference with people I don’t know talking about things I don’t know and suddenly I feel like a baby again. Depending on how powerful (and healthy) a motivator this is for you, you could let it guide some career decisions. For example, when you no longer feel the need to prove yourself at a job, it may be time to either move or take on new responsibilities. This was part of my motivation for moving into management, for example.

Another motivation that’s driven me is to do work that matters. For some, this might mean “putting a dent in the universe” in Steve Jobs’ famous formulation, but for me it just means making some people’s lives a little better. In my first job at Cancer_Care_, for instance, I questioned many things, but never whether the work I did mattered. I can easily contrast that with my next job at a digital agency, where I built, among other things, the transient website for the movie Horrible Bosses 2.

Something else that’s motivated me is the utopian desire to build the perfect system, finally avoiding the mistakes of the past (while of course making new ones). Working at an agency catered well to this motivation, as I’d often be building new systems from scratch. But one of my regrets was that contracts would run out and I’d never stay on systems long enough to perfect them, which led me to my current job at a product company working on the same system for years. However, I’d learn that startups are not great places to seek perfection either, and for good reason. Things change too quickly, including the definition of “perfect” for a system, for that to be worth pursuing. I actualy wonder if that’s the case everywhere.

That’s not to say the desire to build a perfect system is unrealistic, only that if this motivates you, you may need to adjust your definition of “perfect”. If the business’s needs change too quickly, then a perfect system may not be one that meets those needs perfectly at a given point in time, but one that can change as quickly as possible. And of course, if you do really want to build a system that meets a need of yours and never has to change, you can always start a side project.

What’s motivated you in your career?

The Misunderstood Art of Delegation

Tue, 31 Jul 2018 00:00:00 +0000

One of the most-misunderstood skills of a manager is delegation. Before becoming a manager myself, I had the common impression that delegation is when your boss gets you to do their job so they can go take 2-hour lunches. It turns out that’s only half true: delegation is about your manager getting you to do some of their job. But I’ve come to see how delegation is so much more than that: it’s the engine of individual and collective growth within an organization.

How Delegation Works

What really changed my mind about delegation was the Manager Tools: Basics podcast, an essential listen for new managers. In in, the hosts describe the function of delegation within an organization, which is twofold:

Delegation as training: by teaching those you manage to take over some of your responsibilities, you’re preparing them to eventually grow into your job. And in doing so, you free up some of your own time to learn how to do your manager’s job. Done right, everybody is constantly growing.
Delegation as forced prioritization: when your manager delegates one of their responsibilities to you, your plate is presumably already full, leaving a few options:
- You can delegate one of your responsibilities to someone you manage (see point 1 above about growth)
- You can invest in automating one of your responsibilities.
- You can “delegate to the floor” and decide (presumably with your manager’s input) that one of your responsibilities isn’t important enough to keep doing.

Regardless of which you choose, each of these options makes your team more capable and efficient.

Of course, this can go pretty badly wrong, most commonly when someone delegates a responsibility to you and you don’t in turn delegate one of your own: then you just end up overwhelmed and burnt out. So it’s important that everyone on your team understands how delegation works. As a manager, when delegating a responsibility to someone, be sure to talk about how they’ll handle it, and what they can perhaps delegate or drop to make it possible.

And when delegating, you have to be okay with the fact that the person you delegate a responsibility to probably won’t do it as well as you would. Just remember that they’ll quite probably do as well as you did when you were first assigned it. Take the time to teach and prepare them, certainly, but you also have to learn to let go.

Helpful Managers and the Trap of Delegating Up

Camille Fournier, author of The Manager’s Path (another recommended resource for new engineering managers), recently wrote about another delegation anti-pattern: what I’ll call delegating up. That’s when a manager, often in a well-intentioned attempt to be helpful, takes on additional responsibilities from one of their direct reports. As a new manager suddenly responsible for the success of others, I often stepped in to fight my peoples’ battles, somtimes at the expense of my responsibilities to others I manage and the team at large. Interestingly, Fournier learned about this pitfall from the way some management-experienced reports avoided it for her:

when people bring me problems, my first instinct is to think of ways I can help them with these problems. Can I escalate it to one of my peers or my boss? Can I talk to the difficult employee for them, and try to improve the situation? Can I review the project plan and find the areas that are lacking detail and likely to cause the timeline to slip?

As I’ve managed people with a lot of management experience themselves, I’ve noticed that they very rarely take me up on these offers. Instead, they tell me exactly how they are thinking about tackling the situation, perhaps listen to a few bits of advice from me, and then only ask for my direct intervention when their efforts have failed. It’s totally awesome! And it has taught me a lot indirectly about effectively managing my own time, and what good delegation looks like.

Fournier goes on to suggest some useful tips for avoiding the trap of delegating up:

If you can help, timebox your support. Offer to help them think through a decision during your next 1:1, but avoid taking on “homework” yourself.
Ask them to help you help them. If they don’t know where to start with a task, ask them to make a first attempt for you to provide feedback on.

Perceived and Actual Responsibilities

In thinking about delegating responsibilities this week, I recognized a related quick win: dropping perceived (but not actual) responsibilities.

Where I work, I’m pleased to say employees are trusted to manage their own time off. We’re responsible for letting our teams know when we’ll be out and making sure we don’t leave anyone hanging, but there’s no formal approval process for time off. Nonetheless, a new-ish employee I manage, quite likely from having to do so at previous jobs, had gotten into the habit of asking my approval for vacation requests and time off for doctors’ visits. After a few times of responding with “Sounds good!” just to move it forward, I finally took the time to make clear that this wasn’t necessary. They got the message, and that’s one less uninspiring administrative task we each need to bother with.

I see two lessons here. One is to consider how many of your perceived responsibilities (things people ask you for) are actual responsibilities (things you have committed to doing) and to be clear, to yourself and others, on the difference. The second lesson is about the value of autonomy on teams. We were only able to avoid the time-off-request dance and focus on more important things because of the culture of trust and responsibility on my team, for which I’m grateful.

Reflections on My First Few Months as a Manager

Thu, 05 Apr 2018 00:00:00 +0000

A few months ago I accepted a new role at Lessonly: splitting my time between building software and managing 5 of my colleagues. It’s been one of the most challenging experiences of my career so far, but at times pretty rewarding. At some point in your career as a developer, you’re likely to be given the same choice, so I want to share what it was like for me in the first few months before the curse of knowledge sets in.

Why did I choose to become a manager?

The impact I have on the people around me will last longer than the impact I have on the technology, as my colleague Ross wrote movingly about here.
My team was growing quickly, with all the growing pains that entailed. I liked the folks I work with more than I liked building software 100% of the time, so I wanted to help make things better for them (and myself) on the team.

What Do Managers Do?

In a discussion group I started for others interested in management, we began by reflecting on the managers—good and bad—that we’ve had in our careers. Something that became clear from the start is that many managers have very different ideas about what their job is. Micro-managers, for example, think they are responsible for every detail of the work their direct reports do. Absentee managers think their job is to respond to requests from their direct reports but otherwise ignore them. We all agreed that the best managers do two things: they take a personal interest in the growth and development of the people they manage, and they provide frequent, specific feedback about what their reports should keep doing well and what they should work to improve.

If you’re new to management, or ideally before you consider a role in management, you should ask your boss what impact she’ll expect you to have. (Even if you’re not going into management, it’s a fun conversation to ask you manager what they think their job is!) Most managers have some formal responsibilities like performance reviews and approving time-off requests, but don’t make the mistake of thinking that’s all there is to it. Ask how you’ll be measured: will you have hiring and retention quotas? Will you be responsible for the performance of the people you manage, and if so, how will they be measured? A budget to manage? Regardless of how much direction your boss can give you, have frank conversations with those you manage as well to find out what they want or need from you.

If you’re lucky, from these conversations you’ll end up with a clear idea of where to start as a manager. But as you advance in your career, you’ll increasingly be expected to figure out what you should be working on. For me, this was a little uncomfortable. As a software developer, I was used to being given (or asking for) clear direction on what I was to build and why. As a manager, it’s as much my job to discover what needs to be done as it is to actually do it.

New Challenges

The lack of explicit direction all the time is one of the challenges of moving from individual contributor to manager, but there are many others.

Time Management

The hardest one for me was prioritizing what to work on. As a manager, you suddenly have a lot of important-but-not-urgent responsibilities like monitoring your team’s performance and long-term planning. But there’s also no shortage of urgent work, such as helping your team and reviewing code. If you respond to every urgent request immediately, you may never have time for the important, longer-term stuff. I find it helpful to schedule time on my calendar for that to ensure I get to it.

Self-Awareness

Another adjustment is becoming aware of the example you’re setting. Culture arises from the behavior of everyone on the team, but especialy folks in leadership positions. As a manager, if you respond to email while on vacation or interrupt people’s work to ask them questions, be aware you’re communicating to your team that behavior is okay and even expected. Of course, you can also use this to your advantage. By not working while on vacation, you’re telling your team that they’re expected to unplug and enjoy their time off, and you’ll have a happier, more productive team as a result.

Confidentiality

Something else to navigate as a manager is confidentiality, as you’ll likely have more private conversations than before. In a 1:1 for example, someone may open up about a difficult conversation with a colleague, and the next time you talk to that colleague, you’ll have to not let on what you know, in order to respect their privacy. Or your own manager may ask your thoughts on promoting someone, and if you see that person feeling down, you may want to tell them they’re being considered for a promotion, but of course you don’t know that. Particularly if you value transparency, it can feel a little inauthentic to suddenly have secrets to keep, but that’s what it means to be trustworthy. That said, be clear on your ethical obligations. No one can ever ask you to lie, and if someone confides in you information that could hurt others, you have to let them know your plans if you feel obligated to share it. Thankfully, I’ve not been put in situations like this, and hope I never am.

Giving Feedback

As a manager, you have to get comfortable telling people how they can do better. It can be awkward, especially if you haven’t built a strong relationship with the person, but I’d be doing someone a disservice if I see somehow they can improve and keep that to myself. The best way to make this easier on everyone is to practice by telling people when they do well. Then, when you need to tell someone how they can improve, it doesn’t seem like you’re only focused on their flaws. I’ve learned that all feedback, both praise and criticism, must be specific (“Great job!” is bullshit.) Tell someone what they did, what effect it had, and why that effect helped or hindered the team. For example:

I noticed in the last pull request you reviewed for so-and-so that you did some memory profiling and discovered a bottleneck. That’s awesome! Your attention to detail helped improve their code, and with the memory issues we sometimes have in production, it’ll make the app more stable as well. Thanks!

And when being critical, in addition to being specific about what happened, be clear about your expectations, and ask for confirmation. For example:

I wanted to check in about the feature you’re working on. You said it would be deployed it last week and it wasn’t, and now Marketing is scrambling to adjust their communication plan. It’s okay when things get delayed, but I need you to tell me in advance so I can communicate that to other teams, so their work isn’t negatively impacted. Will you do that?

Also, focus on facts, not feelings, since you’re more likely to agree on facts. “I didn’t like how you handled [some situation],” doesn’t tell the person why they mishandled it or how they could have done better. My team is responsible for doing great work, not for making me happy all the time, so I try to avoid suggesting otherwise and leave my feelings out of it.

Conclusion

If this sounds like a lot of advice, it’s not because I think I’ve got management figured out. Since taking on some management responsibilities has been such a challenge, I’m grateful for anything I think I’ve learned that’s made it easier. Still, I’m sure I’ll look back on what an incomplete picture this is before long, and any wisdom that does hold up to time is likely something I cribbed from one of the following sources.

My Favorite Quiet Places to Work in Indianapolis

Sun, 01 Oct 2017 00:00:00 +0000

After reading Deep Work, a compelling case for the power and necessity of focus toward doing our best work, I began to realize just how distracting the modern office—especially the open-plan ones—can be. So I decided to experiment with working from somewhere else. I began searching for quiet, public spaces in the city and was surprised and delighted to discover these gems.

The Holliday Park Nature Center

Holliday Park is a treasure: if you ever want escape to the woods without leaving the city, it’s the perfect place. The Park has several charming hiking trails along the White River and adjacent gorges. I like to park at the Blickman Education Trail off Meridian just north of the White River and hike into Holliday Park from the east.

The Nature Center pictured above is laid out around a bird sanctuary (which the local chipmunks and turtles take advantage of as well). With ample, comfortable seating and an onobstructed view of the birds, the main viewing area is a wonderful place to work. If the chipmunks playing becomes too distracting, an adjacent room holds the Nature Library with a view of the surrounding woods. I find it’s usually unoccupied, and often stow my headphones to enjoy the sounds of the birds outside.

Amenities include restrooms and water fountains, though Wi-Fi is notably (some might say thankfully) lacking, so be prepared to tether or let your coworkers know you’ll be off the grid.

The Irwin Library at Butler University

Built in 1963, Butler University’s main Irwin Library is a beautiful example of mid-century modern architecture both inside and out. The first thing you notice upon entering is the central atrium and fountain around which the stacks and study spaces rise. I loved exploring the library, which has a diverse array of reading and work spaces scattered throughout. My favorite was the desks and comfortable chairs on the top floor with a view of the campus. And the carrels are perfect when you want to focus solely on your work.

Amenities include restrooms, water fountains, Wi-Fi, and vending machines, as well as the restaurants and shops you’d expect from a college campus within walking distance.

The Visitors’ Pavillion at the IMA

Nestled deep in the woods of the IMA’s 100 Acres sculpture park, the Ruth Lilly Visitor’s Pavillion is an incredibly cool building. With floor-to-ceiling glass windows and skylights behind a sculpted wooden ceiling, the main room fills with natural light filtered through the forest canopy above. Stow your headphones if you like—with bird feeders outside, you can hear birdsong among the other sounds of the woods.

Delightfully isolated, it can be hard to find. Whether coming from the museum itself or (my preferred way on bike) the Central Canal Towpath, enter the 100 Acres across from an orange footbridge and follow signs through the woods. (If driving, the 100 Acres has its own parking lot off 38th street.) The 100 Acres are full of art and wonderful to explore if you need a break from work or someplace to eat a bag lunch.

Amenities include restrooms and Wi-Fi, and the museum itself (a walk across the bridge) offers water fountains and a café. My only disappointment was the desk chairs which, like most modern concept chairs, are as uncomfortable as they are cool, so be sure to get up and walk around from time to time. However, there are comfy leather chairs too if you don’t mind your laptop living up to its name. One word of advice: perhaps avoid sitting directly across from the entrance. As I was the only one there, visitors often assumed I was on staff and walked up to ask me questions, and I didn’t know enough to be helpful!

The Central Library

The Indianapolis Public Library System has 24 branches throughout the city, all of which are great places to work (I’m especially fond of the charming Spades Park Branch). But by far the largest and most impressive is the Central Library. Designed for reading, after all, the Central Library has a wealth of distinctive, quiet places to focus, from carrels to ball chairs. I particularly enjoy the desks along the windows (pictured above). On the top floor especially, they’re isolated and provide gorgeous views of the city downtown.

Amenities include restrooms, water fountains, Wi-Fi, a café, print/scan/copy services, and of course if you need a good book, they’ve got you covered. Public libraries are the best—please support them!

The University Library at IUPUI

IUPUI’s main University Library has views of the campus along the canal below. It’s busy, but large enough that you shouldn’t have trouble finding a quiet place to yourself. There are carrels for isolated focus, as well as more public study areas if you want to to be surrounded by others hard at work too. If you prefer silence, the third floor is markes as a quiet floor with conversation discouraged. And the top floor has some lovely views of downtown.

Amenities include Wi-Fi, restrooms, water fountains, and vending machines, and you can easily find a meal within walking distance on campus.

The Law Library at IUPUI

While much smaller than the main University Library, the Ruth Lilly Law Library at IUPUI’s McKinney School of Law has become one of my favorite places to work. Surrounded by the bound artifacts of centuries of legal scholarship (not to mention law students hard-at-work), I find the Law Library especially encouraging of deep focus and attention. The second floor (a quiet floor, with conversation discouraged) is home to a gorgeous reading room (pictured above) with wooden tables and shelves, natural light, and views of downtown. The next floor up offers tables and carrels for a more isolated work environment.

Amenities include restrooms, Wi-Fi, a cafeteria, and an outdoor seating area that’s great for eating a bag lunch on a nice day.

In Sum

If your team supports it, I can’t strongly enough recommend getting out of the office to work for a day now and then. Indianapolis offers a wealth of free, public options for working by yourself, or bring your team along! And if you don’t live in Indy, consider the public libraries and parks in your city, as well as the fact that most public (and many private) universities open their library doors to the general public during the day.

Since I live on the North Side and work Downtown, I know the spots I’ve focused on are skewed toward those parts of the city. Do you have any favorite spots I’ve missed? Let me know in the comments below!

Tips for Naming

Mon, 20 Feb 2017 00:00:00 +0000

Naming things is famously one of the two hard things in computer science. The humor in this adage derives from the fact that naming seems like it should be easy. And yet, most of us can relate to the importance and difficulty in coming up with good names for objects and concepts in our code.

That’s why I really enjoyed this episode of Software Engineering Radio interviewing Peter Hilton on the topic of naming. Hilton clarifies why naming is so important:

Names are about what we mean, not just what the code does.

Names are how we communicate our intention to other developers. When we need to change some code to fix a bug or add functionality, it’s essential that we understand what the code was initially trying to do. Good names can have a huge impact on whether or not we’re successful at that.

That’s why Hilton recommends rename refactoring before submitting any code: it’s the safest, highest-impact refactor there is for improving the long-term quality and reliability of your code. Quite simply, before submitting your code for review, look at all the names you’ve given classes, modules, functions, and variables and think for a moment if there’s a clearer, more precise name. As Hilton says:

Don’t give up. If you try at all you can improve any name.

Better yet, have someone unfamiliar with your code do the same.

Don’t Give Code a Bad Name

Hilton gives some clear signs of bad names I’m sure we’ll all recognize:

Meaningless placeholder names like “foo”.
Incorrect names, such as calling a shipment a consignment, or a function “save” that does not actually save something.
Inconsistent names, such as calling the same thing 2 different names in 2 places.
Vague verbs like “get” and “do”—they may be true, but more specific verbs like “fetch”, “calculate”, and “derive” better convey intention. Likewise with vague nouns such as “manager”: opt for more specific terms like “builder” or “calculator”.
Single-letter names are generally bad unless extremely conventional (such as “i” for an iterative index). Even then, they force an additional step between reading and understanding.
Too-literal names like “appointment list”, prefer domain terminology like “calendar”. Likewise, “employee” is a better name than “company person”.

What’s in a (Good) Name?

These are harder to prescribe than bad names. As Hilton puts it, “Good names are good in a context”. You notice bad names when you trip over them, but good names are transparent. However, one guideline is that good names generally map to what the people using a system would call them. That means go talk to domain experts to find out what are good names. Not only will this produce clearer, ore consistent names within a program, but gives developers a leg up in communicating with the rest of the company, because we’ll be mapping our own mental models to the ones used outside our team.

More on Naming

For more on this, you can watch Hilton’s talk, How to Name Things: the solution to the hardest problem in programming

Henrik Nyh has a good tactic, “The Pairing Test”: he suggests either writing or simply thinking of a list of class or function names on one side and a list of responsibilities on the other. With good names, it should be easy to pair each name with its responsibility. This highlights the fact that names must be judged in the context of other names, not just alone.

And Avdi Grimm has a great episode of Ruby Tapas on naming in which he demonstrates how naming is a process of discovery. Giving a name to a concept or component previously unnamed can bring new clarity to a system:

In my experience, it’s often the case that as soon as we identify an as-yet unnamed concept that’s implicit in the code, multiple beneficial refactorings seem to cascade naturally from the change.

In light of this, naming may be one of the most important things we can spend our time doing.

Using Ecto Schemas with Database Views

Thu, 12 Jan 2017 00:00:00 +0000

Database views are a great way to provide a simplified abstraction over complicated data logic. They provide a way to treat a complex query as if it were a simple table, and with no extra work, you can build Ecto schemas on top of database views to bring the power of that abstraction into your Elixir code!

In a side project of mine, an Elixir/Phoenix app for tracking daily habits called (imaginatively) Habits, the most complicated data logic involves figuring out “streaks” of check-ins. You can check in to a habit (i.e. record that you’ve done it) once each day, and a streak is the number of days in a row that you’ve checked in. The intention—based on Jerry Seinfeld’s “don’t break the chain” motivational strategy—is that if I see that I’ve flossed 100 days in a row, I’m more likely to floss on the 101st.

I don’t store “streaks” as a unique thing in the database. I only store habits and check-ins, but the database can figure out and tell us the streaks if we ask nicely. Unfortunately, asking nicely looks like this:

WITH start_streak AS (
  SELECT
    check_ins.date,
    check_ins.habit_id,
    CASE WHEN check_ins.date - LAG(check_ins.date, 1) OVER (PARTITION BY check_ins.habit_id ORDER BY check_ins.date) > 1
      THEN 1
      ELSE 0
    END streak_start
  FROM check_ins
  ),
  streak_groups AS (
    SELECT
      date,
      habit_id,
      SUM(streak_start) over (PARTITION BY habit_id ORDER BY date) streak
    FROM start_streak
  )
SELECT
  habit_id,
  MIN(date) AS start,
  MAX(date) AS end,
  MAX(date) - MIN(date) + 1 AS length
FROM streak_groups
GROUP BY habit_id, streak

For the purpose of this post, it doesn’t really matter what the query above is doing, only that it returns data that looks like this:

habit_id |   start    |    end     | length
----------+------------+------------+--------
       5 | 2014-04-23 | 2014-04-24 |      2
       6 | 2015-12-05 | 2015-12-09 |      5
       3 | 2013-07-25 | 2013-07-25 |      1
       2 | 2015-09-23 | 2015-09-24 |      2
       8 | 2014-09-26 | 2014-09-29 |      4
       5 | 2013-09-22 | 2013-09-22 |      1
       6 | 2015-09-20 | 2015-09-24 |      5
       8 | 2014-12-04 | 2014-12-04 |      1
       8 | 2016-10-23 | 2016-10-26 |      4
       5 | 2014-02-10 | 2014-02-10 |      1

For every “streak”, it tells me which habit it belongs to (which makes it easy to query streaks for a given habit), when the streak started, ended, and how many days it lasted. Awesome! Except that if I want to use this data in an Elixir function—perhaps to query the longest streak for habit—I have to copy and paste this big ball of SQL into an .ex file, manually set up join()s, etc. Wouldn’t it be great if Ecto could just treat streak data with the same elegant interface I can use with Habits and CheckIns? Happily, it can!

Creating the View with a Migration

The first step is actually creating the view in the database. This requires some SQL like CREATE VIEW my_view AS followed by your query, which can be accessed with the name my_view. As with any database changes, we’ll make this with a migration:

$ mix ecto.gen.migration create_streaks_view

This will create a file in priv/repo/migrations. Unfortunately, there aren’t yet any great tools like Thoughtbot’s scenic library for Rails that make working with database views easier in Ecto. (Update: keep an eye on this issue.) So we’re going to have to write some SQL, but this is the only time. We can run raw SQL in migrations by using the execute function.

Because different SQL is required for creating and destroying database views, we’ll need separate up and down functions defined, instead of the default change. The migration will look something like this (see the entire file on Github):

defmodule Habits.Repo.Migrations.AddStreaksView do
  use Ecto.Migration

  def up do
    execute """
    CREATE VIEW streaks AS
      --- the SQL from above, omitted for brevity
    ;
    """
  end

  def down do
    execute "DROP VIEW streaks;"
  end
end

Finally, running mix ecto.migrate will create our streaks view in the database. With that, we can query the streaks view as if it were any other table:

SELECT habits.name, MAX(streaks.length)
FROM habits
INNER JOIN streaks ON streaks.habit_id = habits.id
GROUP BY habits.id;

          name           | max
-------------------------+-----
 Write in Journal        |   5
 Evening Stretches       |  45
 Floss                   | 607
 Morning Stretches       |  27
 Exercise                |  14

(Elixir: great for oral hygiene!) Of course, I did say above that we wouldn’t need to write any more raw SQL. Since database views behave a lot like tables, let’s see what Ecto schemas can do for us.

View-Backed Ecto Schemas

There’s actually nothing you need to do differently! By default Ecto’s schema macro can use the name of a database view just like a table, and allow the full range of type declarations and associations:

defmodule Habits.Streak do
  use Habits.Web, :model

  @primary_key false
  schema "streaks" do
    belongs_to :habit, Habit
    field :start, :date
    field :end, :date
    field :length, :integer
  end
end

I’ve set @primary_key false because my streaks view doesn’t have a primary key (such as id, which Ecto will expect by default). However, this line won’t be necessary for views with an id column.

As with any schema module, this new Habits.Streak module is a great place to encapsulate logic related to streak data, such as finding the longest streak:

defmodule Habits.Streak do
  # schema, etc.

  def longest(queryable \\ __MODULE__) do
    queryable
    |> select([s], max(s.length))
    |> Repo.one
  end
end

to be used like so:

iex> Streak.longest
607

or, after setting up a has_many :streaks, Streak association in the Habit schema:

iex> Repo.get(Habit, 1)
...> |> Ecto.assoc(:streaks)
...> |> Streak.longest
5

For context, I put the entire set of changes into a pull request, which you can see here: github.com/stevegrossi/habits/pull/4

Elixir In Action

Thu, 24 Nov 2016 00:00:00 +0000

I just finished Saša Jurić’s excellent Elixir in Action published by Manning. I’ve been learning Elixir for about six months now, having learned the basic concepts and syntax from Dave Thomas’s Programming Elixir, which is a great introduction to writing Elixir. But I still lacked an understanding of how to build with Elixir, specifically how to create, structure, and deploy OTP applications, set up supervision trees, and leverage the power of concurrency and distribution. That’s where Elixir in Action was the ideal next step.

After an opening section introducing Elixir (which you can probably skim if you’re already familiar), Elixir in Action is structured around building a to-do list application. That may sound common, but what’s exceptional is how Jurić iterates: starting first with an extremely simple version using only Elixir functions like spawn and receive to send simple messages between processes, then building a generic process abstraction which is eventually replaced by Elixir’s GenServer behaviour. We manage these processes manually until there are so many it becomes a hassle, and so Jurić introduces OTP supervision trees. What we end up building is a robust, scalable, fault-tolerant system running on multiple servers with a database, HTTP API, and caching layer. But the incredible thing about Elixir in Action is how we get there: through a series of small steps from doing-it-ourselves (so we understand what we’re doing) to making it simpler or better with Elixir and OTP tools (so we understand why).

Thanks to Elixir in Action, I feel like I’m finally beginning to grok the Elixir mindset (which it gets from Erlang). A couple key takeaways:

Processes are bottlenecks. Within a process, it does its work sequentially, moving from one task to the next. So generally, you’ll want to keep core processes lightweight and move slow operations out of the way and into other processes. But you can also use bottlenecks to your advantage, such as directing all operations on a given database record through a single process to ensure those operations happen in order and never conflict.
PIDs are impractical. Name your processes. It’s a fact of life that processes die, and while supervision trees can help with starting new processes to do the same job, they’ll have a brand new PID which you won’t know. I never understood why Elixir and Erlang have so many tools for process registry—Process.register, Erlang’s :pg2, :global, and :gproc, and Elixir 1.4’s new Registry module—but once you build an app of any complexity, you very quickly need help keeping track of all your processes. Elixir In Action does a great job of explaining what each one does and when you’d want it.

So whether you’re new to Elixir or already familiar, and want want to learn how to build consequential, highly-available, and scalable systems The Elixir/Erlang Way, then I highly recommend Elixir In Action.

(You’re also invited to the next Indy Elixir meetup, where I’ll be sharing some of what I’ve learned from the book!)

Optimize for Deletion

Fri, 04 Nov 2016 00:00:00 +0000

I just watched Greg Young’s talk The Art of Destroying Software. It’s a compelling, unconventional argument for changing the way we write code to maximize our ability to delete it. Young’s rule of thumb is 1 week: don’t write any self-contained part of your system that would take more than a week for someone unfamiliar with it to rewrite from scratch.

The maxim to prefer lots of small components to fewer large ones isn’t new. Young acknowledges that this idea goes back decades to the Unix philosophy and the intent behind Erlang’s lots-of-tiny-processes actor model. (Marvin Minsky’s 1988 The Society of Mind suggests our minds are even structured thus.) And yet we don’t.

Teams inherently want to build programs that are too big and too complex.

I wish Young’s talk went into the reasons why we so often fall into the trap of writing programs that are too complex to manage—or, more typically, why we let our small programs grow into unmanageable beasts.

Young gives a compelling summary of the benefits of structuring large systems as many smaller programs:

Smaller units lets your team work faster: you’ll spend less time trying to figure out a unit’s responsibilities.
Fewer bugs, as you can fit the entire unit in your head and be reasonably sure of what it will and will not do.
New hires can be more productive sooner, as familiarity with the entire system and all it’s dependencies is no longer a prerequisite for productivity.
Easier deploys, as you can be more confident of the new behavior of a 1000-line change than you can of a 10,000-line change.

Of course, optimizing your code for deletion brings additional benefits when you actually start deleting code:

Technical debt becomes less of a problem. It matters less how great a piece of code is or how ideal an abstraction if you can just burn it down and rewrite it within a week.
You no longer need to predict the future. Developers aren’t psychics. No matter how good you are you can’t know where your business will be and thus what your codebase will need to do 6 months from now. Deletability means you don’t need to build a model that meets your needs 6 months from now. You can delete it then and build a new model once you know what it needs to be.
Deleting code pays off. If deleting code is easy, your team won’t be afraid to do it. That means you won’t get stuck maintaining, waiting on tests for, and ultimately paying for code you no longer need. A friend of mine joined a project for a big company and discovered they were hosting an entire data service for $25k/mo that literally no one used. Deleting code pays off.

Again, Young’s advice is deceptively simple:

Write small manageable programs that coordinate to get a job done instead of writing one big lump of crap.

That we don’t suggests this is easier said than done. But something that’s personally helped me is to develop a visceral distaste for complexity and coupling. Some of this comes from pain felt working on horribly complex systems, and some from the pleasure of building small, simple side projects.

Most programs don’t start out as colossuses. We get into trouble when we don’t realize something is too big until it’s too late to easily split up. So here, guidelines like Sandi Metz’s rules can be the canaries in your coal mines. Sure, 100 lines is an arbitrary limit for the length of a Ruby class, and you should feel free to break it when you have a good reason. But if you at least get in the habit of stopping and thinking about your reasons for growing a class past 100 lines, I’m willing to bet you’ll have very few 3,000 line classes to manage.