Learn caching

Dear new developer,

Caching is a common architectural pattern that helps with performance and scalability. Spending some time learning about this will help you build better systems and understand existing architectures.

What is a Cache?

At the most fundamental level, a cache is a secondary store of a set of values which pulls values from a primary source. There are many reasons why you might want such a store, but a common one is that retrieving the data from the primary datastore is expensive. It could be costly because it is an expensive calculation, because it requires a network call or disk read, or for some other reason.

To avoid the cost of the retrieval, store the value in a cache, and then, when it is needed, retrieve it from there.

Cache Types

There are two main ways to categorize caches.

  • How the data is stored
  • What is stored

Cache data can be stored the same way any data can be stored. In practice, for most applications, you’ll be looking at a few options:

  • Memory
  • Disk
  • Abstracted

Caches that are stored in memory are quick (yay!) but ephemeral (boo!). The cache can go away at any time (whenever the server maintaining the memory is shut down), which means that you can lose access to the cached items at inopportune times. In addition, a memory based cache competes for RAM with other elements of your system. In general, most systems have less memory than disk, which is an alternative storage mechanism.

Retrieving items from disk is slower than memory. Of course, this is true for all disk access, not just that of cached data. The upside of a disk based cache is that you can store far more information.

You can often switch between memory and disk easily and transparently to your application’s code; consult the cache’s documentation to learn more about this option.

The last type of cache is ‘abstracted’. This means you use a cache as a service and don’t really care about the underlying implementation. This could be part of a framework (rails caching, for example), a standalone program like memcached or redis, or a full fledged service like a CDN. Browsers are a cache too. You don’t have to care about the details of this abstraction to take advantage of it. However, you will need to have some understanding of the cache behavior and performance when you operate your software.

Cache Keys

Most caches have a key and a value. The value is the cached value. The key is something that can be constructed by a client to get to the value. There is an implicit communication between the process that puts the value in a cache and the processes that retrieve a value from the cache: they have to agree to how the key gets constructed. Otherwise the cache readers won’t be able to actually read the correct values from the cache.

If you have product data that you are caching, you might have a key of product-<productid>. For a product with the id 15, the data would have a key of product-15. The product prefix namespaces the id in the cache so you can have multiple different types of objects cached (categories, deals, etc). The product id (15) needs to be known by the client to get the data.

Evictions

At some point, your cache will run out of room to store new items. At that point, it will need to get rid of some old items. This process is called ‘eviction’, and there are multiple ways to configure a cache to evict old items:

  • Least recently used (LRU): evict the items that were accessed furthest in the past.
  • Least expensive. If you are caching calculations because they are difficult/expensive to perform, evict the least expensive calculations first.
  • Oldest: evict items that are the oldest. This is also known as first in, first out (FIFO).

Sometimes it can be hard to judge which of these will be the right fit for your system, especially since you may not have a good grasp of usage patterns initially. Generally LRU is a safe choice to begin with.

You should also think about how to trigger evictions. This might occur if the item being cached is materially changed. An example is a new logo for a website. You might want that to be cached in a CDN for years, because it rarely changes and is used everywhere (so you might set the cache period to a long time). But when a new logo is included in a launch, you want to ensure it is used. One way to manually trigger a cache eviction is to use a new name for the logo file.

You might also want to evict a cache item when the underlying source of truth has changed. For instance, if you are displaying the price of an item, when the price changes in the underlying data, you are going to want to display the new cost right away. So when the price changes, you’ll want to force-evict any cache you have. You don’t want someone thinking they can get the product at $X, only to find they are charged $X+1. Not a fun conversation with the customer.

When Should You Use a Cache?

Caches optimize access but introduce complexity. A system is always easier to understand if you pull from a single source of truth.

Introduce a cache when the performance and scalability benefits are required. Otherwise you are simply prematurely optimizing. How do you know if they are required? Evaluate the performance of your code. You can do this by inspection and reasoning about the code. It can be easier to run tests with the cache both disabled and enabled if it is easy to integrate. If you don’t see big changes in performance, there is probably a different slow area of your system.

In addition, think about how expensive a retrieval from the primary datasource is. This will change over time based on your usage and the type of request. You’ll need some metrics to make a good decision as well as to determine if the cache actually helps. Yes, you can introduce a cache and have things be slower, especially if you are on a resource constrained system or you misconfigure it.

Also, consider how much effort it will be to introduce caching. If it is easily supported in your framework or you can add some headers to your HTTP response, caching can be simple to introduce. If it requires you to stand up a whole new system and refactor clients to use it, it can be difficult.

Finally, there are some caches that are built into systems already. Take some time to learn about them; there may be easy wins by tweaking the configuration. Linux has caching as do most databases.

Conclusion

As you might have gleaned from all the examples I mentioned, caches are everywhere. Learning about them and how they can be used is worth your time.

Sincerely,

Dan

Influencing outcomes through output

Dear new developer,

“Outcomes over output” is one of my favorite letters on this blog. In it, Mark, a guest poster, talks about what business actually values (outcomes) over what developers often focus upon (outputs).

Outcomes are things like:

  • more sales
  • happier customers
  • publicity
  • less customer churn

Outputs, on the other hand, are things like:

  • more features
  • test coverage
  • flexible code
  • customer retention analytics

The reason that I focus on outputs is that is what I can control. Outcomes, except in rare cases, are outside of my control. For instance, at a startup a while back, I wrote a lot of code, but couldn’t control how the sales process was going. I felt whipsawed between building things as fast as I could when sales seemed to be happening at a snail’s pace.

It’s important to recognize that business, like life, is a long game. You are the result of choices you (and those who have influenced you) have made.

So, also, are the outcomes upon which you are judged in your career. To square this circle and relate outcomes to outputs as best as I can, I do the following:

  • Recognize the tension. All I can control are the outputs; I can only, at best, influence outcomes. But people pay for results, not for effort.
  • Find best practices. If others have completed tasks or created outputs which produced desired outcomes, learn about them. I don’t follow the same paths uncritically (cargo culting) but need to know about them to make informed choices.
  • Be methodical. When I choose to produce an output, make it good. How good depends on the audience and possible payoff: a script to load data once deserves less care than code that will be run thousands or millions of times. But, I strive to make anything I do excellent.
  • Find ways to incrementally improve outputs over time. These can include automated tools, checklists, and incorporating new practices. I can also choose to stop. This is a strategic choice that shouldn’t be considered every day; once in a while, though, I ask myself if I need to keep producing this output.
  • Experiment. Tweak outputs and see if the outcomes change. Sometimes this is easy and quick, other times I’ll need months to see changes. This requires patience.

What can you do if you are in a situation where the outcomes and the outputs are totally divorced from each other? Where your effort isn’t going have any effect on desired results?

That’s a tough place to be. I would feel ineffectual and frustrated. I think if you are in this situation, you need to sit down on a weekend and spend some time thinking about how you want to spend your life. Of course, there are valid reasons to hang tough:

  • Desired outcomes might be shifting
  • New outputs are being tested
  • Your role is changing

If that is the case, hang on if you can; sometimes a job is stormy but it is better afterwards and therefore worthwhile to see through. After all, outcomes aren’t just hard for you to influence, they are hard for everyone.

But if you do some thinking and find that your efforts have been disconnected from outcomes for a while, and that your nerves are frayed, change the situation. Whether that is leaving a job or moving to a different town, changes can reset your life. And hopefully put you in a position where your outputs are more in line with desired outcomes.

Sincerely,

Dan

Create Value for People

This is a guest post from Minh Pham. Enjoy.

Dear new developer,

I want to start off by saying Congrats and Good job. If you’re reading this, it’s likely you know how to code – and even if you’re still working on getting that first job, that means you have one of the most desirable skill sets in the world today. I congratulate you because getting here took work. You weren’t born with this knowledge, and even if you felt like it came naturally, it was still a journey of discovery, learning, and practice that got you where you are today.

As you look towards your first job – I want to offer you a single piece of advice that may act as your career’s guiding north star:

Create Value for People.

When you have the power to create anything, you begin to realize the importance isn’t on the code you’re writing but rather why you’re writing it in the first place. What value are you creating through your skill? This is why companies hire people like yourself. They are seeking out individuals who can ultimately deliver value to their customers, particularly through software. As you mature, you will realize that much of engineering has little to do with how fancy your solution is, and instead has everything to do with what problem it solves for the user. Once you accept this, you’ll begin to see that discussions of tech choice and code structure rarely matters outside the context of what business value it represents.

This is where your focus should stay.

Obsessions with patterns and algorithms don’t serve anyone’s mission by themselves. Ignore the constant pressure to assert yourself through syntactic cleverness and obscure trivia. These things don’t matter. These things don’t drive value for anyone. No matter how many “experienced” engineers tell you these are important, I promise you no company hires people simply for them to recite principles and algorithms.

While coding might be your latest skill set, it is by no means an engineer’s only skillset. Remember that at the end of the day, it doesn’t matter if your code is ugly, fancy, verbose or concise – the value you create matters. Strive to be an excellent communicator, a quality teammate, and an outstanding human. These attributes will guide your engineering efforts to ensure you bring value.

No matter where your career goes, if you focus on creating value for people, opportunities will never be in short supply. Desire for specific skills may rise and fall, but people will always look to those who can create value.

With that, I wish you the best of luck and may our journeys cross again,

Minh Pham

Minh Pham believes you should lead how you want to be led. This has been the guiding principle of his career since he started. As an Engineer, he always wished he had someone who would guide him – telling him what’s important, what he has to work on, and what he should ignore. Having gone through all that and then some, Minh now looks to be the positive influence he wishes he had.

As a manager, Minh’s greatest passion was teaching people the skills to create and drive the careers they want to have. Now as a career coach, he works to show people they have the power to build the life they want.

Minh believes anyone can do it – and he promises it doesn’t involve linked lists or graph traversals.

Sometimes you just have to ship it

Dear new developer,

For me, there comes a point at the end of every project where I’m just sick of it. I’m sick of the project. I’m sick of the technology. I’m sick of project management system. I’m sick of the code.

Sometimes, I just want to see the whole thing burn. Or better, just ship it.

Now, I think that there are two solutions to this problem, and which one you pick depends on your timeline. The first is to take a step back. Talk to a team mate. Work on something else. Talk a walk. Take an extra half hour for lunch.

This may give you perspective to help you dive back in and add just a bit more polish. That polish, which may take the form of additional UX refinement, testing, or even wordsmithing the help messages or marketing text, can help make the project shine.

That’s what I call ‘running through the finish line’ where you want to leave it all on the field. That doesn’t mean you don’t make compromises or that you won’t revisit decisions, but it does mean that you do the best you can. Sometimes to put in that final effort, you need to take a break.

The other choice is to just ship it. This is a good option when you are up against a deadline. It also helps if you know you are a perfectionist and/or afraid of putting your work out there. Nothing is perfect and if your work never sees the light of day because you can’t accept that, the world is losing out (as are you). Finally, it can help if you take that time off, acquire that perspective and know that you’re done with this phase of the work.

I just published a book. It was released on Aug 16. I’m very proud of it, but there were times when I was just plain sick of it. I ended up taking some time away from it and that helped me make sure it was the best book it could be.

When you are working through the final bits of a project, sit back and get that perspective. And then, ship it!

Sincerely,

Dan

The best career advice I’ve ever gotten

This is a guest blog post, lightly edited, from Josh Doody. Enjoy.

Dear new developer,

Let’s talk about jobs.

My first job

I was 25, and I wanted to move my career along as quickly as possible. I had my first real job, and had gotten three raises and a promotion in only two and a half years, nudging my salary up 12% from when I started. I was feeling pretty good.

Then two things happened that changed my career’s trajectory. First, my boss told me that striving for big raises and promotions would get me nowhere. “The way things work around here,” he said, “is you might get a big raise one year, or even two raises, but they’ll eventually work out so that you’re right back at the average. It’s hard to get ahead or fall behind.” He meant that I might be on top now, but eventually I’d regress to the mean. My boss had been working at the company for 30 years, so he knew what he was talking about. Ouch.

Second, I had been begging for a new challenge and just hadn’t gotten one. I had been looking for ways to stay interested, but got more and more bored, so I asked my boss for a new challenge. He eventually offered me a new opportunity: the same work I had been doing, but in a crummier location. I asked if we could keep looking.

A couple of months later, my boss came to me with yet another opportunity: I would move to a different building and redraw schematics for a 20-year-old piece of test equipment. I could hardly believe it, but he’d actually found a worse position than the one I’d already turned down.

But he made it clear that I couldn’t keep saying no to these opportunities—I was being too picky, and it was making both him and me look bad—so I reluctantly accepted the position.

I moved from a building where I had tons of friends and a 10-minute commute to a building where I had no friends and a 40-minute commute. The clock was ticking on my time at that first job.

Looking for something new

I started looking for a new job for two reasons: first, I needed to get out of there; but second, I wanted to know if I was over-valuing my abilities. I was young but self-aware enough to realize that one very plausible explanation for my frustration was that I just wasn’t very good at what I was doing. Maybe my boss was in the uncomfortable position of having a really ambitious, but really ineffective employee on his hands. Maybe he had done everything he could to pacify me without putting me on an important project that I would just screw up.

Around this time, a friend reached out asking if I knew any web developers looking for work. I told him that I did know someone…me! I had a little bit of web development experience and after we talked, he suggested I might be a good fit for his startup’s client services team. I interviewed with the company’s CCO and he offered me a job as a Project Manager. I had also been interviewing for a managerial position in a different city and, although I didn’t get that job, those two opportunities reassured me that I was a valuable enough employee to take seriously. I happily took the Project Manager job at the start-up.

Before I could start my new job, I had to wrap things up at my old job. Ironically, my manager on my temporary project (redrawing old schematics) had been the best boss I’d worked for my whole time there. On one of my last days, as we were looking over some schematics, he gave me the best career advice I’d ever gotten:

Josh, your first job is where you get your first job. Your second job is where you get experience. Your third job is where you get paid.

My second job

My second job meant a career change from electrical engineering to project management. I took a small pay cut, but that was completely reasonable considering I had no client-facing experience. I went on to work there for five years, scratching and clawing my way to a slightly higher salary than I’d been making as an engineer.

Of course, the money wasn’t what I was after—I wanted experience, and I found it by pursuing unusual opportunities, including a “special project” that ended up being crushed after nine months and getting me laid off. But they soon hired me back to work in a different capacity, which again provided great experience for slightly less pay. This position marked my second pay cut since starting my career, and I was making exactly the same salary I’d made as a test engineer, five years earlier. But by then, I had amazing experience and was in a position to move nowhere but up.

After five years in my second job, I finished up my MBA, which I’d been pursuing on the weekends. I decided to quit and take some time off to travel, relax and recharge.

My third job

After my hiatus, an old colleague from the start-up reached out from a different company: “You looking for work?” I was, and my experience landed me my third job, where I would make almost 30% more money than I had been making when I quit the start-up eight months earlier. Just like my sage manager said, my third job was where I got paid.

I’m glad I heard that advice during my first job, because it allowed me keep putting in time when things got tough. I knew that getting paid was inevitable if I continued to do good work and gain experience. That advice allowed me to stop obsessing over raises and promotions and start focusing on trying new things and building my resume so that when I did encounter a lucrative opportunity, I would be ready.

Sincerely,

Josh

Previously published on JoshDoody.com

Josh Doody is a salary negotiation coach who helps experienced software developers negotiate job offers from big tech companies like Google and Amazon. He also wrote Fearless Salary Negotiation: A step-by-step guide to getting paid what you’re worth to help software developers navigate job interviews and salary discussions to earn more throughout their careers.