StreamSparkAI

speaker 1: Thank you. And hello everyone. Good to see you here. I'm radolav geororgiv. It's really hard to pronounce Eastern European name, so just call me rado. It will do. I'm the CEO of software development company, where we mainly do Jango and react. This is like our bread and butter. So if you want to find more about me and those beautiful people over there, this is hasoft team. We went to jangokin Heidelberg this year, and if you happen randomly to PaaS by us while we were taking this picture, you're going to say those people are from Eastern Europe. Definitely. Yeah. Let's talk about scaling images. So the red arrow pointing at Istanbul, this will create some historic turmoil. But Bulgaria is right next to this rtaro, and we're somewhere over there in the United Kingdom, so it's far away. But in case you don't know where Bulgaria is, and the final thing that we do is we do a lot of free courses for students, mostly Python and Jango, where we prepare people for their first job. We have like a highly successful rate of placing people in companies, and it's pretty good. So but we are here for something different. We want to talk about Jango. So let's see by raof hands, who here is working with Jango daily. All right? And now the people who are working weekly, but just weekly, not daily, they had to touch jungle from time to time. All it's all pretty good, Ali. Hope this is going to be interesting and actually valuable and resourceful for you. So what do we want to achieve? First thing, we want to enable productive Django teams. As mentioned in the keynote this morning, it's really hard when you have like a big software and it's not using a popular framework, and it's really hard to sit up in bootstrap. It's really hard to get people to contribute because there's a high barrier to actually running any piece of code. So this is one of the things that we want to do when we're developing Jango applications, and we're developing Jango applications that are going to be used for like next five to ten years, and we need to support it. And we need to add features, remove features, fixed books, write books and so on, which is the same thing. We want to have a stable software and we don't want to introduce random bugs, which we are going to see you in the presentation. And also avoid any kind of needless abstraction. Avoid this thing where someone writes a really nice piece of abstraction and only he or she understands it, and then everyone else who has to touch this code is like, Oh, just ask him, or better, make him or her write this piece of code, because we have a needless abstraction. So this, we are going to try to eliminate this. And the most important thing for me is to have a repeatable pattern. And this applies not only to Jango, but applies to generate to software development. The best thing we can do is to have a repeatable pattern of adding new features or fixing existing features or removing existing features. We don't want to invent the wheel every time you have to do something new. And this is what we're trying to achieve today. All right. So the main question, because I'll need an entire day of jangokhan to talk about the entire topic, because Jango is a big framework. So the question that we're going to focus on today is where to put business logic in Jango? And the first thing that we need to ask ourselves is how do we define this mythical beast called business logic? Any suggestions? How do you define business logic in less than five sentences? I'll try. So for me, this is everything that's specific to the domain of the software that we're writing. If we're writing a software that deals with hotel management and booking, everything related to how we book and manage a hotel is a business logic, or more generally speaking, the constraints and the relationships in our cult. Every time you see an if, and this if is not in an utility method that checks some strings, this if probably is a business rule that needs to be applied in the software. So almost everything that's not framework or utility is business logic. It's the thing that we're writing the software about all. So let's talk about boxes. I really love this how to say example framework or frameworks in general. They give us boxes, and those boxes are places where we can put our code in. We have boxes and in Jango, we have several boxes where we can put and structure our application and actually put our business logic, because our business logic is like 80% of the entire code base. The rest is views, models and so on. And the very first box that people actually hit when writing Django is the models. And I think models are like the big and integral part of Jango. And we kind of use Django because it has a really nice aurso. They defined the so called data model. We have like entity relationship diagram. People love creating those diagrams. I don't know why and we have to turn this diagram into models with the proper relationships, with the proper constraints. As I said, there are a very important part of jangle and they define the relations that later we're are going to use in a so called business logic. And models can have properties and static or class methods. And we usually start putting some small part of our business logic in those model properties or in those model methods, either class methood, static methood or instance methood. And the thing where people start tripping over and things start to get like bad is when they start putting business logic in the save method. So this is a lot of tutorials do this. It's like you really find safe method, and you start adding code there. You start adding tasks that send email, mail, and you run a test tuwhere. You bootstrap a lot of users and you have production settings on your lomachine. And while running your test tuit, you send emails to randomly generate emails, which is pretty fun experience. So models, let's see an example. Let's have a course model that defines a starting end date. So business logic can be this thing here has started or has finished. It depends on the now we have some get now that returns the proper date in the proper time zone. And we say, or false, the course has started or course has have has finished. And we return this in some api and the ui updates accordingly to this. So this is an example of business logic, is pretty straightforward and is the right place to add it. Another thing that models allow us to do is to define custom validation, because sometimes the validation needs to be expressed as a Python code and not just as a database constraint. And this custom validation is usually written in the clean method. And the thing that we usually do is redefine safe kfu clean, not to forget to call it in the other place and raise validation errors if something strong. And this is also a good place to put your business logic. It's like additional validations and models, keep those two things together, properties and additional validation. And this is kind of ugly, and it's, according to me and this presentation, not the right way to do it. So don't try to read the code. I've just pasted some call from a system that we use for our our learning ering management system that we use for the courses. When you create a course, you need to generate weeks for the course because we organize content around weeks. And this suddenly starts to feel heavy because I have a really core thing defined in my model, safe. It's going to work, but it's not going to scale. The thing is, if I put it here, I have to put everything else related to this here, tasks, morlogic, whatever you imagine related to course creation. And for me, models should take care only of the data model, have simple properties and define extra validation. And this is where we draw the line. We don't do like everything is going to be in the model. We just don't do this because then we get a fat models, which don't scale very well because you have to put everything there. So business logic in models, three simple rules that we have is like it's okay to do it in properties. And we'll see later when it's not okay, it's okay to define additional validation in clean and it's totally not okay to start adding a bunch of coating your safe meod because you want to keep some kind of separation of concerns models, data model, not business logic. All right, Carry on. The next integral part of Jango is views and apis. I'm going to talk about apis using rest framework for us. Rest framework is Jango. You install it as a separate library, but it's the go to library that you're using when doing apis. The same thing complies for views. I Sadon't have enough time to cover everything. So the views and the apis there, like the http interface to the rest of the world, and they call things on the inside and can hold business logic, which is usually shown in tutorials. Take a look at your tutorials. It's like if you don't do it in the models, you're going to do it in the post method of your api or of your view. And as I said, I will focus on apis. This is copy pasted example from rest framework, which again, is a great framework. And the thing that's being defined here is two apis that do a Croot over some snippet model. And even more, if you scroll, if you scrolls down in the tutorial, you're going to end up with this example. This is a model view set that needs two things to work, a query set and a serializer. You can then use a router, and you have a cruot api over a model in around five lines of coal, which is pretty neat. It looks really good. But then what's happening? Do you know three lines of code? Where in the three lines of code is your model being created? For example, if you use the create api, who knows who can shout, raise a hand where in the serializer. So what happens? You hit this api, the create model mixing from the rest framework, create metal is called, the serializer is instantiated and validation is run. And then the snippeserierizer method, creits code. So we have three lines which look neat for an api, but then the heavyweight needs to be done by the serierializer. And it's a third party library thing, which is really nice, but it has to create our object. So if you want to put additional business logic around creating some something which is like almost all of the cases, this is really good for hooking people up. Look, three lines of cold cruot, we're awesome framework. But then reality comes and you have to do five additional things when you're creating a snippet. And then you go to the serializer and you really find create method and you start dumping cold there. So let's talk about serializers. They're like really, really great. And I wish Tom somehow forks them out of rest framework as a separate library, because the plane serierilizer is really nice bottle serializer junrelated plane. You can use them for a lot of things. So they're great to transform Python and rm objects to json or whatever you want, texml and so on. Jason is the usual thing. They are great to do. The other thing, transform Jason coming to an api and make it Python data or even or m object data, make the query for you. And this is where we draw the line. Two things that serddes are really good at and should be used. It's okay. The last thing that people often do is to use serializers to create objects. It's like you define create, and you have like 500 lines of create in a serializer. And it's, again, a separation of concerns. Sting, serializers, the first two things, not the third thing. It's not a serializer 's job to create your object. It's your job to create your objects. It's not the apis job to create your objects deep in some obstraction because you have three lines of code. It's again, your job. All right, so here's an alternative. Then you say, rto, okay, I won't do it in the safe method. I won't do it in the serializer, but I will definitely do it in the post of the api, which is also shown in a lot of tutorials. Again, dump, just a dump a codump, which not even sure if it's correct. We need to refactor this, but this is what people also do. And this may be okay if you think hard about it. There are not a lot of bad reasons to do it. And then again, business logic in apis is great until you need to do the same business logic in a view. You have api for reacand, you have some old Django view with html, you have to do the same thing or you have to do it in a task. You have salary and you have to do the same thing in a task, or you have to do the same thing in a command, or you have to do the same thing somewhere internally in your system because you have a chain of automation where snippets or courses are being created as part of other flows. And then we have five places where we can do the same thing. And the the bad approach here is to start instantiating the api to use the post method in order to have the same business logic. It's like red flags everywhere. So if you have only api, perhaps it's okay. If you have something very simple, perhaps it's okay to use the model view sets of the rest framework. But we're talking about projects that evolve and scale with time, and we need to be flexible and we need to have teams working on them. So this is not the box we are looking for. None of the things that I've shown so far. It's not the box that we need for business logic. And oftentimes we need more boxes. How to say it's wrong to assume that all the things that jangle giving us are all the things that we have to use. And we try to put and we try to put things to boxes where they don't belong. And when we do this often and and time goes by, then we end up with a bad design, and then we end up with random books, and then we end up with unmaintainable code based because you have a serizer somewhere in your code base, and you just add a simple property because your api needs it, and you break five other apis because five other apis are using the same serierilizer and you don't know that. And such things can be really, really bad developer experience. So we need a new box, and this box is like it's pretty simple. A personal note on obstruction. I don't feel really good when I have three lines of code and then 100 lines of abstraction underneath that creates my object in the database. And then I'm not sure what I have to redefine. I have to read the rest frame work code, which is nice, done it many times, but I don't feel good when there's a lot of layers of abstraction between me and the orm, which actually saves the object to the database. That's why I don't like the model view sets or the generic apis or views at all. They should not be used unless you have a really simple project. And on the other hand, sometimes when reading things from the database, you have additional business logic. You have to do filtering based on other objects in the database or filtering based on your internal business logic. It's not like just the least api view, which dumps all objects, define pagination, and we're done. It also happens. We can get away with abstraction here, but when the thing comes that we need more flexibility and control, we have to, again, redo everything. Those are the existing boxes that junkis giving us models, views, templates, tasks for resalery, for example. And none of those boxes are suitable for heavy business logic. I don't have time to show you some nice template examples where you put a lot of business logic in templates and you do a lot of queries and you can't understand from where these queries are coming. But templates are like the worst idea to add business logic, just no time for this. So the thing that we're doing at our company, since we writing Jango every day and all of our clients or our projects, whereas in Jango, we're developing a style guide, which we are constantly debating and updating because we want to have a repeatable pattern of writing Jango apps that is nice, easy to use. It's not a burden. It's not something that you have to learn as an abstraction. And that's why I'm here actually to share where we put our business logic. And the first new box that we introduce, which is the fancy name service, is called services. It's like services pmodule in your app. Each app has a services pi. You can split the module inside however you want, but this is the general unit that deals with the business logic, meaning creating rm objects, doing additional stuff with those rm objects, calling color services and so on and so forth. We'll see an example in a second. And the art thing that we're doing with the services is we have like a specification. It's a keyword only function with type notations, which helps a lot for documenting what you're actually doing with this function. Speaks the domain language, can call other services, can call other tasks. It's like all the heavy lifting is isolated in a core part called services and selectors who see them in a second. And the rest of the Jango is either communicating with the service like apis, or being communicated from the service like rm. Yeas, I said, can handle everything that needs to be done and works mostly and mainly with models because Django without your ram, it's like no use to use Jango at all. So here's the good stuff. Here's the example. Pretty simple, service code create user. As you can see, we're doing the we are doing the keyword only thing, which means you can't call a service with ten positional arguments and wonder which is switcher or what is what. You have to PaaS the keywords by name explicitly. This is the first thing. The second thing is we have type annotations. What are the arguments? What is the return type? And even if we don't use my pi yet, it still needs something for jangle support. It's really nice because it lets you understand what the service is doing. It lets you jump around the project. I see that this is returning to user model, and I want to see what the user model is. I hit some combination in my editor or ID de, and I jump there and I see and I return. It's really, really nice. And what's happening in scientists? I create a model, and I call two other services. The first one creates a profile. The second one calls the task. Again, we don't have time to talk about services and salary tasks because there's a lot of interesting stuff there. But the thing that we do is we wrap over tasks in a service because it this way is isolated nicely. We can mock, we can test, so on and so forth. All right, so this is it. This is a pretty simple service. Here is a little bit more complicated service. When creating a course, I need to generate wigs. So this is the right place to do it. A bunch of codes can be tested. We'll talk about testing in a minute. And the rule of thumb is that every nontrivial operation that touches the database should be done in service. It's like you should have no orm code in your api, in your model properties. Talk about this in a second or anywhere else except your services and selectors. This is the core that defines how your software is behaving. And on top of that core, you have the shell, which is Jango, which communicates with the core, exposes api to the rest of the world. And this should I will probably add something, link to Gary Bernhart. Talk about boundaries. It's really nice. And he did it on rails Conf way back in the time. And this is actually the thing that we doing, because inspired this by this stop and services, take care of writing to the database selectors all the way around. Take care for things that need to be fetched from the database. Again, selectors, pi, we have let me, let me get all the bullets. They take care of the business logic when fetching, which usually some kind of filtering they can call arbitrary services, selectors or tasks, whatever needs to be done, and can handle permissions, filters and so on. So permissioning filtering should be also happening inside the core and not on the shell, not from the api. Of course, there are some exceptions, again, which we don't have time for this. And here's a simple selector from a project that needed this simple selector, you want to list all users for a specific user, for a specifically logged in user, which is called feg by, but there are restrictions from the business, from the domain about what you can actually see. And this code was firstly written in a get of an api and was not really scalable because after this, all other interactions for fetching from the database needed some kind of permissioning and filtering. So selector, you say, give me the visible users, the ID, so I can filter by them so you don't accidentally see something that you don't have the permission to see. And as you can see, having a plain Python called just using the jangle orm is much better than having an api. And then this is like the asterisk star on the model properties for using business logic, if your model property is a simple function on several of your fields, it's good if your model property starts making queries outside of its relations or even with its relations. And let's say something like this, this is a model property. It better be a selector because right now we have business logic. We have not present students is making queries. And the general the general roof thump here is if you have a model property and you just add it in the list api and you get m plus, one problem, it should be a selector. This is a bit too deep in Jango, but if you have a property that's make and okay, then you're going to say select related. Okay, this can solve the problem. But if you add a property which cannot be solved with just select related to be to solve the m plus one queries program, then it should move to a selector. So on our roof thumb, and then you're going to ask rado, but what about the apis? We started with the three line api moof set, and now I'd like to imagine some huge apis or how our apis are going to look. And the good thing about our apis is that they're going to look absolutely the same. They're going to be five to seven lines long, and we're going to use zero abstraction over them, which is actually the same thing that we achieve with model view sets, but without the abstraction. So there is no hior complex. And the thing that we achieve with the apis is this repeatable pattern. Because if you open or pick a random api in a project that follows strictly, this is going to do the same. You're not going to be surprised. Or if you are surprised, then there a good exception to everything. And there should be a good comment why this is a good exception from everything. And you suddenly see the boundaries between things, and you suddenly start to navigate the code base much better, because you know what to expect this wiyou use frameworks for. All right, so let's make a course, Croot api. First we have the list api. We'll will talk about why we define serializers inside in a minute. We have a get. We use a selector, we use the serializer to transform the result to something that json serializable. And we return three lines. That's all, nothing more, nothing less. Detail up. All right. We have get, but with course se ID, we use a selector, which will probably fetch or the other thing that we can do here is to use an input serierilizer that's using primary key related field that's going to fetch this course se ID for us. There are like nuances and options here, but we call a selector and we return and all list and get apis. They look like this. You can call generate them if you want. There's nothing in this api that's actually hidden or complex. It's a stupid thing that calls a selectro. Again, create api. We need to create objects in our system. For list and detail, we definine output serializers. For create an update, we define input serializers. And what we do get the serializer validated. We probably need some additional mixthings to catch validation error. And to now this won't result in a 500. This is actually okay because it comes from refrawork. Never mind. We call create core service, which usually just duplicates the serializers as keyword arguments. So I can just expand it with two stars and then return whatever I want, create it. So on ten minutes, remaminute, thank you. And finally, an update api, which looks basically like create and detail. We use update courses service and we use input to your lizer again. And this is for me, it's also neat, but it's neater than the other thing, because there is no hidden abstraction. There is no additional knowledge of the framework that you need to have. And let's talk about the elephant in the room, which is not me, the serializers. We nest our serializers inside the api, and we do this because otherwise it's really easy to reuse serializers. I really can't say this word serializers, all right? And if you reuse serierilizers, then someone can add or remove something from a base serierilizer that's inherited five times down the road and break who knows how many apis, which if you don't have a good integration test coverage, the users will catch it. It's the user driven development where they do the final testing. So we try to inline serializers and yeand, try to not reuse them. It's like, have you need a good reason to reuse? And we still haven't figured out how to freeze a serializer that's being reused. It's like you're reusing it, but you cannot modify it anymore because you're going to break apis. And this happened to wait too many times, and that's why we started doing it. We have a rule of thumb, which says for output serializers, it's okay to use model serializzers. Sometimes it saves lines of code, but for input serierilizers, it's strictly forbidden. You don't use model serierilizer because this kind of says, let the model serierilizer do the creating. Now it's a plain serierilizer, and you define all the fields that you need. No model serializer there. And this saves a lot of problems and a lot of headaches. This simple roof thumb. And to avoid reusing, sometimes you kind of want to have, like in ruby, anonymous classes in Python in order just to create in place an anonymous class which extends a serializer. And I can define few fields. Sadly, we don't have the syntax, but we have the tools to do it. So we have this really neat utility method called inline serierilizer, which takes all the keywords for a serializer and defines a fields as a dictionary. And then there is no need to reuse a weak serierializer defined elsewhere. You can just do it here in place, in line. I don't know why I've put the implementation here. It just creates a new type from the base serizer in the resources to this presentation. There is a code for this. It's pretty neat. It's pretty neat to use it and to end everything. Let's talk about testing because it's really important when we do this service selector thing, the car and the shell, testing models comes down to testing the properties and testing the clean method. And oftentimes testing properties that are functions on fields. You don't have to instantiate models, fast tests, good tests, also clean. You don't have to instantiate, you don't have to instantiate models if your clean method just uses fields, some set of the fields and do something around them, which is really nice. Testing services and selectors. This is the heavy lifting. This is where you actually hit the database. You use a lot of you use a lot of mocking because you use isolate different parts. For example, you've tested the service a lot and you want to test just the integration between four services. You mock the services that you've tested or you mock the email sending that we've done three times to randomly fake generated emause. A lot of mocking here. And it's like the heavy lifting down here. Your brain capabilities go here and testing kpi's. There's with this approach, there's usually no need to test apis because they're just an interface and it's usually delegated to integration. Test five minutes from an correct, usually delegated to one of my last sites, delegated to integration testing, which is also really nice. But my talk is not about testing. If you want to hear more about Jungo testing and what approaches we use, you can listen to Martin who here in Lemmer. Muir, well, how do you pronounce this lemon Muir? All right. In this hole at six, at 405. All right. So to recap and to finish, avoid business logigin models, safe method forms, any kind of forms in any kind of serializers, just don't do it again. Views in apis, there's no need to do there. Business logic and templates, taand utility methods. Utility methods should work like pure functions on pure Python data structures. On Python data structures, you should not have a utility which speaks your domain, or if you have it, don't call it a utility. Just create a simple function with leading condor score and don't use it elsewhere. So utilities should not be doing business logic. Again, tasks. Tasks is an interesting topic. Sometimes you have to do it. I don't have the time for this story. Selectors and services, again, use services for creating update actions. Tions, use selectors for list and detail actions. It's really simple approach, but it pays a lot in the long run. There's more to it. I've just published this morning this style guide because we have it internally and we're going to update it a lot. So I'm going to tweet it, add it to the presentation, it's already added and so on, in case you're interested how we're doing this. Again, I'm not saying this is the best way to write Jango because there are many ways that are actually valid and you can build software with them. This is just the way that's working for us, and it's working from experience. It's working from shipping software that's being used by a lot of users every day, and we have to be flexible with the features. So that's it. Thanks. Any questions or welcome? Hello, I usually have my business logic in the managers. Like I extend the bulk Update Manager and have the business logic here. Do you think this is a bulmethod? So you say you put your business logic in custom managers and custom query sets. That's actually good, but it's not the best. How to say it's too or and specific. You can achieve absolutely the same thing with services and selectors, with custom managers and custom query sets. But this kind of makes you want to use them in apis and tags and templates and so on. So it's basically the same, but we just define a nice boundary around it. It's still good to, if you feel comfortable doing things in managers, just wrap the thing that you've done in a manager in a service and call the service. Just have one entry point to the database and one entry point outside of the database that that's the general idea to have the boundary. So Yeah, I hope this answers it. Very nice talk. Thank you. In all the examples we saw, everything seemed to work out okay. Are there any best practices for how you handle exceptions raised from within services and transactions within these services? Yep, let me let's hope I've copy pasted the writing. I missed it. All right, so now let's talk about exceptions first since we're doing validation and permission checking in these services and selectors, we usually define a mixin that goes in all of the apis that just handles the exception that we define as in our core in our business and just translate ated to rest framework exceptions. This is how we handle it to avoid 500. And then for transactions, we usually if you have a service, you decorate it with transaction atomic, that's it. And if there are special edge cases where we have some special edge cases like you need to be sequential in some ordering, for example, invoice numbers you lock, but you do this in the service. Yeah. I was wondering about now you have all the logic and selectors and services, that's also what do you think about then providing shortcuts to those from a model, for example, as a property? And if yes, what do you do about the circular import problem? This is pain. So circular imports, you either import the module and use the dots, or you import it inside the function, which makes it really hard for for testing. So I'm not sure I could give you a general solution for this, because I still struggle every time I hit a circular import. As for shortcuts, we usually do the other way around. We have some methods and properties that are simple enough on a model, and we expose them via service. We do the other way around because again, we try to have the boundary between the api and the database with the business logic in between. We try to have no other way of using the application aside from using a service service elector. Okay. Did you mention that when we have a property that calculates something in the relation we have attended just today, another speech talking about how to optimize the rem to do queries and so on. So what be the best practice to do that calculation as a property? Doing a selector to doing that calculation outside the property and the property call that. Yeah. So actually, this is evil stock right in front of you for the query organization. This is a nice edge case. In case you need this, you better preserve them as properties or you have to use a serizer method fields in order to use the selectors. But this is an edge case. And when there's an edge case, you have to decide we sure gaa team make a decision and add a comment why why we've done this, why we did this and why it's not like the the rest of the system. But it's really this is like context specific. Sometimes you can do it with a selector. It's pretty easy. Sometimes you have to use the cache property because you're doing some annotation, grouping and aggregation in the query set. Okay, thank you. Now, and just to say, we don't follow 100% of this in our projects because we developed this over time. It's like last week, we made some decisions that are looking good, but we try gradually to turn our entire code base to be like that. The last project that I did, which was a two months long project, it was entirely like this. And it works like magic, but it is really good. But we learn along the way also, Hey, I haven't used enough dager rest framework, so I'm just going to double check this. Is there anything like a model form serializer that would basically serialize it to the form that could then get like clean verified that way? So there's a plain serializer, which just takes, for example, json and turns into Python objects. And there's this thing called model serializer, which takes json and turns it to rm object. So Yeah, would you use forms that all with this method? So most of our projects are using, for example, react as a front end. So we don't use forms. But the thing that we had, we had duplicate forms and model serializers. And for me, the best thing is to drop one and just use model serodizers, render test forms because you have, again, duplication. And it's really painful. Thank you. I have a couple of questions. The first one, why not putting selectors into services? Because you get way too many services. You get way too many services, and you make the separation visually in the file. It's like there's a comment in the middle which says things that create and update, things that listen and new details. Yemy gut feeling would be that if I'm able to create something with the service, I should also be able to select it with the same service. I guess. I guess yes. Same goes feeling here perhaps is a naming thing because we started with services which were doing both, and then we decided to start factoring the selectors. So perhaps the entire thing should be called service and we should have subnames inside of it for the two different types of functions. So I agree with you. And then then what's your take on using the main driven design approach towards structuring your project? This is basically the main driven design. It's like you do the data model, you expose the business logic and you do the apis. Yeah but from my point of view, of course, Yeah. Okay. Okay. All right. That was all the time we had for questions. Give another one last round of applause, please.