StreamSparkAI

speaker 1: Okay, we're going to get started. Hello, everyone. Welcome to this talk by Lyn ruod, the design of everyday apis. Lynn has requested that if you have any questions, shebe more than happy to answer them outside of the room. After her presentation, say you can see her there, so take it away. Thank you.
speaker 2: Thank you. Awesome. Thank you for being here. It means a lot that you chose my talk this afternoon. My name is Lynn root. I am a staff engineer at Spotify. So I note I learned blender to do this cool effect just for this. But Yeah, I am based in New York. I've been at Spotify for over ten years now. You might also know me from pi ladies. I'm the chair of the global council of pi. Ladies, thank you. Be sure to check out our booth. We got a lot of cool stickers there. And Yeah, heads up, this is a jam pack talk. So No Q and A. I will be speaking as fast as I can, but I'm happy to chat afterwards in the hallway. All right, so jumping in, about ten years ago, I bought an electric kettle from brevel and had like this plug. And this plug, like blew my mind. I know like it's an American plug, and not everyone uses American plugs here, but I found myself asking, why don't all plugs have a hole like this? Honestly, it's probably because of patents, but not only does it make it very easy to like unplug so you don't like wiggle it around and maybe get electrocuted, but it's also very obvious to me how to use it. It allows for easy grabbing and pulling. The simplicity and the thoughtfulness of this design, it made me think, you know, what else am I anknowyingly struggling with? So this brought me to a book called the design of everyday things by Don Norman. You might have noticed this is the inspiration for my talks title. And this nice coffee pot has actually dubbed the coffee pot from masochists. And the premise of this book is about how design serves as communication between object and user, and how to optimize that communication for a better experience. So Norman writes, design is concerned with how things work, how they are controlled, and the nature of the interaction between people and technology. So the range of things is not limited to physical objects like the brevel plug that I'm in love with, or the masochistic coffee pot on the cover, and also includes artificial creation like software and digital interfaces, the layout of a conference room, or organizational structures, etcetera. But what makes design good? So Norman says the two most important characteristics of good design are discoverability and understanding. And so breaking this down further, discoverability is described as, am I able to figure out or figure out what actions are possible and where and how to perform them? The author lays out five key elements to discoverability, and I'm sorry, I'm going to breeze right through them. The first one is affordances. Affordances determine what actions are possible. What it's what a user can do with an object based on a user's capabilities, building on top of affordances. The second element is signifiers. Signifier is some sort of perceivable cue about the affordance. So signifiers communicate where an action should take place. So it's like the bar across a door where you push open. The third key element of discoverability is constraints. Constraints are limitations or restrictions. And they give us clues about how allow us to determine the course of action by limiting the possible actions available to us. The fourth is mappings. A mapping is the relationship between a control and an action. An example of a good mapping would be where you have two lights and then two light switches. The left light would control the or left light switch would control the left light, and vice versa. And the final key element is feedback, where the results of an action is communicated back to the user. Feedback should communicate clear, unambiguous information back to the user in order to be effective. Immediate feedback is ideal. Delayed feedback can be disconcerting and lead to a feeling of user abandonment or failure. So these five key elements of discoverability, affordance, signifiers, constraints, mapping and feedback, they can help build up to the second part of good design, understanding. So understandability means, is it possible or easy for users to figure out how the product can be used? Understanding is developed by forming a conceptual or a mental model of how something works. A conceptual model is just a collection of explanations, often very simplified, so the user has their own conceptual model of how a system or a product works, and the designer designs their system or product using their own conceptual model, and designers expect users conceptual model to be identical to their own, but because they cannot not communicate directly with the user, the burden of communication is with the system and its design. So good conceptual models are key to understandable, enjoyable products, and good communication is key to good conceptual models. The communication comes from these key aspects of discoverability, affordances, signifiers, constraints, mappings, and feedback. And so you might be able to start to see how these ideas, essentially human centered design, can start to apply to software, to designing a library for other people to use. My first instinct when thinking about this is to think about libraries that I've worked with that I may or may not have some fun in using. I'm not gonna to name and shame any Python libraries here. There is one non Python library that and that I do lo that I'll talk about is f of mpeg. It's a well known cli tool that's designed for processing audio and video. This is a screen recording of me running a command to list some documentation. So whenever I use f of m peg, the way for me to figure out how to do something seemingly simple, like creating a gif into a video, or a video into a gif for a slide, I end up needing to go copy commands from stack overflow because I'm am sorry, the tools documentation sucks. It's so powerful, and I know that it can do many things that I can't even fathom. And I will continue to reach for it because of its ubiquity and its performance, but there will always be this kind of reaction when I need to use it. So this made me think, if I am an engineer who develops tools and infrastructure for other engineers, how many masochistic teapots have I unknowingly shipped? How many have I caused? And so you know there are probably much better people to give talks on this, better ideas and theories. So I'm not an authority on api design, but for me, there is a missing connection between the theory of good api design and the actual implementation. So this brings me to why I'm here today. I'm not sure if anyone else has seen my previous talks, but I have a habit of writing talks for myself, ones that I really wish I would have seen many years ago. So this talk is mainly for paslane, a talk that tries to take the key elements of good api design and apply them to a real world example. So as able condense this into three tenants, three key principles that make delightful to work with api. And don't worry, all share a link at the end with my slides and resources. So first I want to set the stage. In a previous talk of mine, I went through building a chaos monkey system with a pub sub queue. I'm going to stick with that and build a library to work with a pub sub quelike service. I'm going to call it chaos quebecause. Why not? It has nothing to do with this car. I just like it. So let's look at some starting code. First, we have a class that defines a message object, and all it does is contain data to be passed to and from a chaos cube. We have a second class, a client that essentially publishes two and consumes messages from our chaos pub sub queue. And this is our starting library. Perhaps you're already thinking of some things that you want to change about this, which is good. But hold tight. In order to develop a little bit of empathy for my future users, I also want to write a bit of user code that interacts with my library. So on the user side, we create a chaos pub sub q client reinststantier one, and then we create a topic in subscription. And then catching if they already exist, and continuing on, then we start interacting with our client. We first have a for loop to publish five messages to our pub, suq ue, and then a while loop to consume those messages. We print the message, maybe there's some debugging going on, and we do some sort of processing on that message data and then acknowledge the message from the queue when we are done processing. And finally, maybe the client has a couple of network connections that we need to clean up and close, and maybe some buffers to flush. So let's improve this for the user and start iterating on our library. Start with the first of my three principles. An api should be intuitive. And note that I mean intuitive for the user, not exactly you, the implementer, or the maintainer. With an intuitive api, the user can be lazy. They don't have to think too hard about how it works. They don't have to remember complicated details because their intuition correctly explains it to them. They don't have to spend a lot of time learning new parts of the api because it all works pretty similarly. An intuitive api builds on the user's preexisting conceptual model and tries not to do anything surprising. Nothing that unnecessarily works against any logical or domain constraints. With that, I have three changes I want na make to our chaos cuue to improve the intuitiveness of the api. So let's start with some low hanging fruit and name the methods of our client class using the domains nomenclature, like publish a message to the queue, pull a message from the queue, acknowledging a message is complete, and drain a queue of messages. So right now we have add message git, message mark, message done, and clear message queue. Let's rename them to publish, pull, act, and drain. I've also dropped the noun from the method names since I found them to be a little bit redundant, particularly because in my mind, the noun that we're operating on is in the function signature already. It's the message. And look, I now have room to indent four spaces instead of two. So why are we doing this? It plays into the mappings element of discoverability. Naming your functions something similar to the domain that they're supposed to work in can help result in immediate understanding of what your api is supposed to do. The next suggested change steps back a tiny bit to think about appropriate abstraction levels. This might be a little controversial. Naming is certainly hard, but what I mean is if you find it difficult to name your classes or your functions, or you find your object names to be a little awkward or clumsy, it's an indication that you might have awkward abstractions. So if you recall the pubsuclient, we have a few curious methods. We have create topic, create subscription, closed client. But this client seems to be managing a lot in one object from the user's point of view. What if we are just wanting to consume some messages and not publish them? Do we have to create a topic for some reason? Will there be multiple connections managed by this client? Some unnecessary. So let's not add confusion to the user and break this client up into two, one that works with just publishing and one that works with just subscribing. This also allows us to clean up the method names a little bit. This then creates natural constraints on what the user can do. If the user must create a topic, they have to instantiate a pub client and are limited to only those actions provided with the pub clienmethod methods. However, our clients also create some unnecessary constraints on our users. Which brings me to my third change. Let's make sure our api has symmetry. We allow the user to create a topic or to create a subscription. What does a user do if they want to change an existing topic or subscription or if they want to delete one? So let's not limit the user unnecessarily and provide symmetry with their methods. Just having a create method without its logical symmetric pair, triplet, or whatever gives a misbelating signifier to the user. You can be certain that users have worked with other apis that provide both a create and a delete method, a get and a set method, an upload and a download method. So it would be counterintuitive to either not provide such functionality or provide that functionality but not follow the convention in symmetrical naming. So some of these changes might seem a bit contrived. It's really hard to come up with a simple, succinct example that's also relatable. But hopefully the message is starting to come through some changes that come later on in this talk could very well apply to this intuitive tenant, and I'm sure that there are changes that I'm not even addressing or applying here. Again, I only have half an hour, but let's move on to the next tenant. An api should be flexible and I mean flexible for the user. Again, not it's a whole other talk for flexible apis for maintainers and contributors, but basically, basically a flexible api lets you do what you want. In my point of view, a flexible api lets the users to get started quickly with the more basic use cases and then allows the user to adjust as they continue on to solve more complex problems. So flexibility comes down to the question of how many problems can users solve once they learn your api? And so for this tenant, I have four changes that I want to make to our chaos. The first is providing same defaults for the most common use cases. So recall that we have some methods that work with publishing and consuming messages. The main resource that these methods work on is the message. The timeout and retry arguments are not required to the core functionality of publishing, polling or acknowledging a message. Also, I have found myself as a user to not know what a good number of seconds a timeout should be or how many retries I should be making. So as an api designer, I should make optional behavior as optional keyword arguments. Maybe 30s is the most common timeout value, or maybe it comes from the default of the service sign. Maybe it's the default to have zero retries. So by moving a lot of optional settings into keyword arguments, this allows the user to focus on what is required of them. This change also goes beyond the flexibility of an api by forcing the use of keyword arguments and providing sane defaults, we limit the number in the order of positional arguments that the user has to remember. So for instance, both timeout and retries are integers. If they were both positional arguments like before, the user could easily inverse them when calling the method, and that would actually safely PaaS tycheckers. Maybe we have many more keyword arguments for our methods, like request ID, verbosity, level, much more. If you have many, many arguments, positional or keyword, you might want to take a step back, because it might be another hint at clumsy or awkward code and abstractions. You might be unnecessarily exposing some complexity. So with type hints in a lot of arguments, if users can't understand your type annotations, maybe you need to rethink and refactor. So for the sake of my slides being readable, I'm going to roll all the keyword arguments into star quargs. But do not do this at home, particularly for your public api. This is because for library, especially with automatic documentation generation, instead of finding the list of arguments and their meeting and documentation, users will see absolutely no information except for a vague, it takes some keyword arguments. And then if your library has no docc strings, which shame on you to begin with, but then the user can't just simply look at the funcsignature. They must dive into the code base to understand what the starquargs are, how they're handled and passed around. So again, quarirks, these starquarks on these slides are just to make them readable. So moving on the second change that I want to make. To increase flexibility of our apis to minimize the users need to repeat themselves, we saw a loop in the user's code calling publish for each message. So in our pub client, our publimessage has one positional argument for the message to minimize forcing the user to repeatedly call publish for each message. We can accept multiple message objects in the function signature by collecting them into star arcs, and then we can do the repetitive work for them. So hopefully you are writing user code as you are building your api, since while doing so, you can look for common patterns that the user code that you can minimize away from the user code, things like loops and repetition. As I said earlier, the tenant of flexibility is meant from the user's point of view. Providing flexibility to the user requires you to be predictable and to be precise in what you do for them. So if you recall, part of building that conceptual model, that understanding for the user is to provide constraints and feedback. So we provide flexibility in what we accept as input. There needs to be a clear constraint in what we return. We can't make the user figure out what object type that they're dealing with from what we give them from our method. So we call our subscription client that returns a consumed message or it returns none. This doesn't seem too precise or predictable. The user has to check if they got a message at all or if they are dealing with none. So what's the nun use case here? It's when there are no more messages in the queue to consume. Now if you look at pexisting apis like Python's queue library, it raises an error when the queue is empty. And so that seems pretty reasonable. So let's update our function signature to only return a message object and then raise if there are no more messages in the subscription queue. And so this goes with what Don Norman said about feedback, that it should communicate clear, unambiguous information back to the user in order to be effective. So this last change I want to make against supports the uers being lazy. Basically, let's not force users to provide data that you generate for yourself. Recall the message object. It has an ID, some data, and a published at timestamp of some sort. Now, when creating a message, there is no need for a user to supply their own unique ID or to tistamp it when it's published. We can do that ourselves. Now, we would probably not have the timestamp be set when the message is being instantiated, and we probably have not generate the ID in the client itprobably come from the server. But coming up with real world examples is difficult to just go along with me. But now, hopefully we have a bit of user understanding. Often the user may want to print or log a message for debugging purposes. For that, we should provide them with a Repper method to make it easy to print out for the instance. So now the user doesn't need to worry about printing the necessary attributes of the message, but just the message object itself. So this is just one approach to redefining our message object. For the user, we could use data classes. This removes some boilerplate. For us, it does get a little clumsy to set defaults, in our case, using post in it. And Yeah, the code kind of goes off to the slide, but it's still nice. A third option would be the third party package, adders, the author of adters, which allows me to go into back to force space and dentation in my slides, but also if I needed adders, would give me more to provide flexibility and constraints for my users, since not only does it give me a clean way to set dynamic defaults, it also has validation converters and the ability to abuse in it, more so if I needed. I'm not a page chill for adders. I'm just a happy user. So if your api is not flexible, if you find yourself saying, I'm sorry, I know our api should be able to do that, but for reasons that you don't care in there within our control, it can't. So if an api is not flexible, users will eventually ditch the library and go to something else to solve their problems. And so for my last tenant, an api should be simple. The complexity of an api can be measured by the cognitive load it requires to actually use it. Complexity hurts our understanding. We've already made some changes to reduce the cognitive load of our api, like consistent and appropriate method naming, limiting the number of positional arguments in our function signatures, and minimizing the amount of repetition a user has to do. But we can make a few more changes to make our api more simple. The first change is to provide composable functions. Apis that follow the mathematical closure property tend to be simple as well as flexible. Loosely, an api holds the closure property when every operation returns a data type that can be fed into other operations. So this means that different operations in your api can be composed together. So for example, in Python, we're able to chain a bunch of string methods together. Since the output of many string methods is another string, this closure property makes it easy to combine multiple operations to get the desired result. Maybe users don't often do this, but following this wherever possible allows users to not have to remember differing funcsignatures operating on the same object. So looking at our subscription client, we have two methods that operate on the message object. It's pretty reasonable to think that once you pull a message from the chaos queue, you'll want to acknowledge it. Otherwise, itget redelivered. So there is no reason for us to be restrictive in our function signature chair for acting and require a string that adds to the cognitive load for the user. They have to remember that they must give the ID of the message that they want to act, and they can't just PaaS. The object that they've been working with. So let's change that to just take the message object. We can know figure out how to get the ID from the message from there. So now it's simpler for a user to just pull the message from the subscription and act it. And now while we're at it, let's also accept the ability for our users to give us multiple messages if they wish to batch up their acknowledgments, and that this certainly adds symmetry to our method signatures and removes unnecessary loops in user code. Now while we're on the topic of loops, the next change that I want to make aligns our api with some Python idioms, like being to be more pythonic. So leveraging language idioms also leans into what programming languages afford. Different programming languages allow you or Ford you different approaches to the kinds of problems that we solve, like supporting reflection and introspection, or supporting default values for function parameters, or how to define an iterator. So looking at our subscription client, again, the user has to call poll to get a single message. And as you might recall, the user code had a while loop. So let's make that easier for the user and provide an itter method that returns an iterator. So providing an iterator is helpful in cases like this where the user is consuming a stream, event of events or data. It's also helpful when users might need to page through results like search a search query with hundreds or thousands of items. This itermethod simplifies users code so that they can just do a simple for loop. As a for loop listens to stop iteration exception, the user is now able to also remove a try and accept further empty queue around their consuming code. And continuing on leveraging the language idioms, you might remember, both clients have a closed method managing any connections and buffers underneath the hood. This forces the user to remember to clean up after themselves, adding to complexity and cognitive load. So you might see where this is going. Let's make use of a context manager, providing an easy way for users to not have to worry about remembering any finalization or cleanup behavior. There's even a third adjustment we can make to align ourselves with existing idioms and habits. We have two create methods in our pub and sub clients that the user would have to handle. It would raise if a topic or subscription already exists. We can follow the convention of other apis like os, Mader or even sql's create table and allow the user to not care if something already exists and just make sure it exists. So we'll add an exists okay keyword and default to false since it might not do what the user expects. So it will force them to opt in. And then the last change, we can think of simplicity as convenient to get started. How much do we have to learn to start to be able to use this library? What do I have to do right now to get it working? If the api is not convenient, if it takes many steps to get started or if I have to page through docs in order to get a very basic understanding, the tools adoption will suffer. So provide your user with convenience. And you do this by treating your read me like you would a newspaper. Give them the most important details above the fold. And if users want to know more, then they can turn the page and read through the rest of the documentation. So when I'm looking at a read me, I want three things above the fold. The first is, how do I get your library? Do I need any system level or non Python dependencies? What are one or two examples that I can copy and paste immediately to try this out? And as a side note, please reconsider if your examples are using the repl. It makes it really annoying to copy and paste those examples into code. Maybe that's just a pet peeve of mine. And then third, where can I go to get more information? So give me those three things at the top, and I'm good to go to get started with your api. So creating a simple api is all about reducing cognitive load on the user, reducing what the user has to keep in their head while working with your api. All right, so that was a lot. Let's rewind. We started here a very simple api that only contained two classes. As we went on, we broke down our code into three classes. And now this slide is a bit less readable, but that's okay because we started with a lot of user code, 29 lines to be exact, and we got it down to just 14 lines where you know all the code can even just fit on a slide and be kind of readable. This should be the goal, not necessarily a number of lines of code, but it's about user empathy when designing a delightful api. So to recap, is your api intuitive for the user? Is it flexible for the use case? Is it simple enough to essentially learn once? You have to keep in mind that your software will become a part of a larger system. So your only choice is whether it will be a well behaved part of that larger system. The how is a little elusive. So hopefully this example in improving a dummy api is helpful. But it's a lot to remember, I know. So if you only remember one thing from this talk besides my cheeking awesome since way theme, then remember this. If all else fails, standardize. Don Norman leaves us with a good catch all. When no other solution appears possible, then design everything the same way. So people only have to learn once. Thank you. We can find all my slides here.