Category: Object Oriented Programming

About microservices and software architecture

This topic of microservices continues to pop-up in my professional activity. 

I remember that some time ago, I heard James Coplien say in a video something like “Microservices…don’t let me start with this”. I wanted to decipher this. Because of that, I re-read his wonderful software architecture book(1) and all of his OOP materials (2).

Small note: I tried to find again the initial video and no luck. Still the words remained in my head. After James reply I realized that I was not off track with my memory.

I have to say from the beginning that this trend regarding microservices is becoming rather dangerous (maybe because of wrong focus or  because of misunderstanding of what software architecture really is or because it is not known what OOP actually is or …). I usually say to my friends, colleagues the nightmare we will have in 10 years or so when we will have to maintain the so-called microservices approaches.

What James Coplien said in that video and how he said it hunted me. Then, suddenly, I have found a way to decipher/decompress what he said when I read the paper “The DCI Paradigm: Taking Object Orientation Into the Architecture World”(3). Below are fragments from the article which triggered my thoughts/perceptions/insights:

● “…The authors have been working on a paradigm called DCI(Data, Context and Interaction) that places the human experiences of design and use of programs equally at centre stage…” ➜ Is hard to do that with microservices, they are not about humans at all. The way the subject of microservices is put on the scene is not about human experience at all.

● “…Architecture is the form of any system created through conscious design, and it thus has strong human elements both in its process and its product.The term form implies a deep mental model of the essence of some Structure…” ➜ With microservices it is hard for me to see the holistic form of the system, that’s why I can’t see microservices as architecture. For me is about some low level technical details and about deployments. 

●“…Form is the deep essence of what is common between these systems, just as Victorian architecture is the essence of common elements across innumerable houses…” ➜ Here is mentioned the Victorian style, but I thought of the Romanian style named Brâncovenesc(4). James and Trygve are right. When I look at the houses, churches I can say that it is about the Brancovenesc style and I can see that it is different from Victorian style. So, when I look at an HR application or Planning events application I should see that form which speaks about that kind of application. And the fact of what iron or cement was used is another story. I think with microservices it is hard to see the forest from the trees.

●“…Why do we do architecture?It might be useful to revisit some of the key goals of architecture. As mentioned above, Vitruvius reduces the purpose of architecture to utilitas(commodity or utility), firmitas (firmness) and venustas (delight). These goals echo strongly in software, which has adopted them with its own emphases. More broadly, architecture is, and always has been about form. Except among specialists, the English word form is often confounded with structure, and software folks in particular often incorrectly infer that a system’s architecture is the structure of its artefact. The proper architectural usage of the term form has historically been more precise. It’s important to differentiate form from structure: Form is the essence of structure. We can talk in detail about the form of gothic cathedrals even without having a gothic cathedral at hand. Form is the conceptualization of structure in terms of the relationship between parts, and between the parts and their environment. Many given structures can implement a given form, just as there are many (different) gothic cathedrals, all of which implement the forms of gothic cathedrals…” ➜ So, maybe, microservices came with a “structural” flavor and also a simple mechanism of sending/receiving data.

●“…SOA defined services, but at a level that was usually far removed from the code; it is probably a better metaphor for urban planning than for the architecture of a house…” ➜ Yes indeed, this SOA stuff was at the far extremity of the code, just a simple entry point in the system, but nothing else. I was amazed to see whole books regarding how to explain these endpoints and how the entire code structure has been influenced and is not ok. 

● “…This may well be because great architectural talent arises from domain knowledge, and it’s difficult to treat architecture as a generic discipline within the (generic) discipline of programming. In the end, architecture has arisen as a generic discipline of tools rather than the result of a quest for beauty and utility…” ➜ The knowing of the domain and its importance is diminished because all the discussions focus on the hammer, scaffold,..

I asked James Coplien to help me out. I really needed his help. He was very kind to review and give me feedback about my insights:<< “Yes” to all of your insights. The main one that bothers me is the lack of a coherent system overview. All understanding is local and the system behavior is taken to be emergent. That doesn’t work except at incredibly large levels of scale and scope.>>(5)

After his feedback I had the pleasure and honor to meet him in person, in Athens. I had participated in 3 of his wonderful trainings about Scrum – a blog post will soon come about this. I hope that soon I will also participate at his DCI training. I am waiting for this DCI training for a very very long time – sometimes I feel this is an impossible dream. Actually, one important reason I decided to have the Scrum trainings with him was the hope to have some minutes with him to discuss about architecture and OOP and it worth it.

(1) James O. Coplien, Gertrud Bjørnvig, “Lean Architecture: for Agile Software Development, https://www.amazon.com/Lean-Architecture-Agile-Software-Development/dp/0470684208/ 

(2) James O. Coplien, “Publications”, https://sites.google.com/a/gertrudandcope.com/info/Publications

(3) James O. Coplien, Trygve Reenskaug, “The DCI Paradigm: Taking Object Orientation into the Architecture World”, https://doi.org/10.1016/B978-0-12-407772-0.00002-2

(4) “Brâncovenesc style”, https://en.wikipedia.org/wiki/Br%C3%A2ncovenesc_style 

(5) Text used with permission (March, October 2019)

How testers can provide value for developers

On the Rapid Software Testing slack Channel(1), one month ago, an interesting  question was put by David Högberg:

“Here is an interesting story and question from my friend in another Slack-group:

(translated from Swedish)

”Today was the last day of a sprint in one of the teams I coach at….

A backend developer has been very productive this sprint, he has pulled in more tickets than planned and finished those. Instead of pulling in another ticket on the last day he took the initiative to work on the unit test coverage and integration test coverage in our most important repository.

Now we have 100% code coverage for both unit and integration (API) and a branch coverage above 90%.

Besides this he always tests his own code, with both wide and deep understanding of the requirements. He always ask for feedback regarding his own testing, wants critical code reviews and practises risk-based exploratory testing.

This is what QA-coaches and testers will have to work with in the future. How do we create value for these future developers?”

….What are your thoughts regarding my friend’s question? How can testers provide value to developers who deeply care about code quality and at the same time are good at testing themselves?” (2)

The point raised by David Högberg is important and should not be ignored. Also, I saw that some testers were somehow speechless regarding this, like there was nothing to say/do/challenge/help.

There are lots of things testers can do. Testers must/can challenge things and there is a lot to challenge. So, for example:

1. The fact that what the developer does are unit checks , integration checks not tests.(3)

2. He said that he reached 100% code coverage for unit checks and integration checks. Actually code coverage is a measure and number for programers only. This is because it helps the developer to see areas not covered at all, not how well that area of code is covered. So, this number should not be exposed to management, because you can have 100% code coverage without code actually being covered. From the moment when you hear that code coverage number is exposed, there might be trouble – I know it sounds strange, but for me is a sign of exaggerated confidence. Maybe I am wrong, but for me is a heuristic for looking with care, because of the things which can’t be articulated in simple boolean confirmation statements.

I have to say that each time when I discuss with managers I try to do my best to convince them to ignore the code coverage number and the fact that it means nothing.

By the way, when I have the occasion I ask programers this simple question: So you are so confident with the unit checks?  If the answer yes, then I ask: Good, are you ready to bet you salary for 3 months that I’ll not find any problems? Usually, I perceive hesitation.

A good technique to verify the actual lines/conditions, in terms of coverage, is mutation checking(4). Even with this technique, I have in my mind the difference between testing and checking! (3)

3. As a tester you can help the developer with the sampling, the data she/he is choosing. And, as you know, this is not a skill which can be learned in a day or two. Sampling at the unit and at the integration level is important and I saw a lot of times unit checks and integration checks which repeat themselves.

I do not mention that testers, need to know very well about cognitive biases, critical thinking.

4. The developer said that he/she made unit checks? Perfect. Challenge that. Take a look at this brilliant article by James Coplien: Why Most Unit Testing is Waste . You’ll notice that this article is not so easy to digest, is deep – it speaks about the theory of information, also it does not follow the current dominant thinking about unit tests. I’m sure it will help you to make that developer think. Maybe negative reactions can take place, but is OK, we want fruitful disagreements.

5. The developer said he/she is doing TDD? TDD does not help at big architecture and OOP. It helps, maybe, at the micro level, but for sure not macro. This means architecture, OOP is fully exposed in a big and nasty ways. Maybe integration checks can help to see that the parts are fitting together, but are not helping to see that they are ok built. What this means? Well… nasty things are hidden at the integration boundaries. Developers don’t do OOP, but Class Oriented Programming. OOP would mean to reflect the mental model of users, which this does not happen. All the work is split between a lot of classes and functions. I think you can imagine the dangers.

Regarding TDD, I noticed that some persons begun to make some research about it and it is interesting: A Comparative Case Study on the Impact of Test-Driven Development on Program Design and Test Coverage or Does Test-Driven Development Improve the Program Code? Alarming Results from a Comparative Case Study

By the way, I also liked this post: TDD is dead. Long live testing

So, please, go and see the research, I doubt a lot of programmers do that…

6. Unit checks and integration checks is also code, this means that is prone affected by things like: bugs, maintenance, design, analysis, architecture.

7. I love the article of Max Boisot with the patterns(5). For example: for 10 dots there are 45 possible links and 3.5 trillion possible patterns. I’m sure in applications there are more than 10 connected dots. Is not possible for a developer, no matter how smart he/she is, to be sure he/she covered all the possible patterns and also to be aware which pattern is more important than other(s).

8. Regarding “A backend developer has been very productive this sprint, he has pulled in more tickets than planned and finished those.” Maybe I’m interpreting this in a wrong way, but sorry… I’m not interested in the productivity of the developer. This is a very nasty stuff. For me is important that the team and the whole work to be ok. When I read productivity of individual, I begun to doubt the good functioning of the team and this is bloody dangerous. Is like measuring tester for the bugs introduced, sorry, but no. Were there any other members actually working on stuff? If yes, why he did not took care to help finish that?  – I’m deducing this from the text, maybe I’m wrong. In Scrum the team works on a single PBI at a time.

Still the list is short. I would have added about cognitive dissonance, of how the team structure affects the code structure, but you got the idea…

Conclusion: So how testers might create value for these kind of developers? I think that when a tester will put this question then is a good sign because learning/helping continues.

(1) Michael Bolton, “RST Slack Channel”, https://www.developsense.com/blog/2017/10/rst-slack-channel/

(2) Text used with permission, thx David Högberg

(3) James Bach, Michael Bolton,”Testing and Checking Refined”, https://www.satisfice.com/blog/archives/856

(4) Robert C. Martin, “Mutation Testing”, https://blog.cleancoder.com/uncle-bob/2016/06/10/MutationTesting.html

(5) “The problem of connecting the dots”, https://sensing-ontologies.com/the-problem-of-connecting-the-dots/