About microservices and software architecture

This topic of microservices continues to pop-up in my professional activity. 

I remember that some time ago, I heard James Coplien say in a video something like “Microservices…don’t let me start with this”. I wanted to decipher this. Because of that, I re-read his wonderful software architecture book(1) and all of his OOP materials (2).

Small note: I tried to find again the initial video and no luck. Still the words remained in my head. After James reply I realized that I was not off track with my memory.

I have to say from the beginning that this trend regarding microservices is becoming rather dangerous (maybe because of wrong focus or  because of misunderstanding of what software architecture really is or because it is not known what OOP actually is or …). I usually say to my friends, colleagues the nightmare we will have in 10 years or so when we will have to maintain the so-called microservices approaches.

What James Coplien said in that video and how he said it hunted me. Then, suddenly, I have found a way to decipher/decompress what he said when I read the paper “The DCI Paradigm: Taking Object Orientation Into the Architecture World”(3). Below are fragments from the article which triggered my thoughts/perceptions/insights:

● “…The authors have been working on a paradigm called DCI(Data, Context and Interaction) that places the human experiences of design and use of programs equally at centre stage…” ➜ Is hard to do that with microservices, they are not about humans at all. The way the subject of microservices is put on the scene is not about human experience at all.

● “…Architecture is the form of any system created through conscious design, and it thus has strong human elements both in its process and its product.The term form implies a deep mental model of the essence of some Structure…” ➜ With microservices it is hard for me to see the holistic form of the system, that’s why I can’t see microservices as architecture. For me is about some low level technical details and about deployments. 

●“…Form is the deep essence of what is common between these systems, just as Victorian architecture is the essence of common elements across innumerable houses…” ➜ Here is mentioned the Victorian style, but I thought of the Romanian style named Brâncovenesc(4). James and Trygve are right. When I look at the houses, churches I can say that it is about the Brancovenesc style and I can see that it is different from Victorian style. So, when I look at an HR application or Planning events application I should see that form which speaks about that kind of application. And the fact of what iron or cement was used is another story. I think with microservices it is hard to see the forest from the trees.

●“…Why do we do architecture?It might be useful to revisit some of the key goals of architecture. As mentioned above, Vitruvius reduces the purpose of architecture to utilitas(commodity or utility), firmitas (firmness) and venustas (delight). These goals echo strongly in software, which has adopted them with its own emphases. More broadly, architecture is, and always has been about form. Except among specialists, the English word form is often confounded with structure, and software folks in particular often incorrectly infer that a system’s architecture is the structure of its artefact. The proper architectural usage of the term form has historically been more precise. It’s important to differentiate form from structure: Form is the essence of structure. We can talk in detail about the form of gothic cathedrals even without having a gothic cathedral at hand. Form is the conceptualization of structure in terms of the relationship between parts, and between the parts and their environment. Many given structures can implement a given form, just as there are many (different) gothic cathedrals, all of which implement the forms of gothic cathedrals…” ➜ So, maybe, microservices came with a “structural” flavor and also a simple mechanism of sending/receiving data.

●“…SOA defined services, but at a level that was usually far removed from the code; it is probably a better metaphor for urban planning than for the architecture of a house…” ➜ Yes indeed, this SOA stuff was at the far extremity of the code, just a simple entry point in the system, but nothing else. I was amazed to see whole books regarding how to explain these endpoints and how the entire code structure has been influenced and is not ok. 

● “…This may well be because great architectural talent arises from domain knowledge, and it’s difficult to treat architecture as a generic discipline within the (generic) discipline of programming. In the end, architecture has arisen as a generic discipline of tools rather than the result of a quest for beauty and utility…” ➜ The knowing of the domain and its importance is diminished because all the discussions focus on the hammer, scaffold,..

I asked James Coplien to help me out. I really needed his help. He was very kind to review and give me feedback about my insights:<< “Yes” to all of your insights. The main one that bothers me is the lack of a coherent system overview. All understanding is local and the system behavior is taken to be emergent. That doesn’t work except at incredibly large levels of scale and scope.>>(5)

After his feedback I had the pleasure and honor to meet him in person, in Athens. I had participated in 3 of his wonderful trainings about Scrum – a blog post will soon come about this. I hope that soon I will also participate at his DCI training. I am waiting for this DCI training for a very very long time – sometimes I feel this is an impossible dream. Actually, one important reason I decided to have the Scrum trainings with him was the hope to have some minutes with him to discuss about architecture and OOP and it worth it.

(1) James O. Coplien, Gertrud Bjørnvig, “Lean Architecture: for Agile Software Development, https://www.amazon.com/Lean-Architecture-Agile-Software-Development/dp/0470684208/ 

(2) James O. Coplien, “Publications”, https://sites.google.com/a/gertrudandcope.com/info/Publications

(3) James O. Coplien, Trygve Reenskaug, “The DCI Paradigm: Taking Object Orientation into the Architecture World”, https://doi.org/10.1016/B978-0-12-407772-0.00002-2

(4) “Brâncovenesc style”, https://en.wikipedia.org/wiki/Br%C3%A2ncovenesc_style 

(5) Text used with permission (March, October 2019)

The hidden dimensions of code review(s) – part 2

Some dimensions will be treated in this blog post and on part 3 of this blog posts series(see part 1 here) , my friend, Ionut Albescu will help out with the dimensions and sub-dimensions which were left out. 

The dimensions

■ Programming(Design, Clean Code, Refactoring, Unit Checking) 

There are a lot of materials covering these aspects. Just search for books by James Coplien, Robert C. Martin, Martin Fowler, Michael Feathers and you will get an idea of each of these topics. Usually, in appearance, this is the dimension were code review is concentrated. I said in appearance because these topics cannot be mastered in a month or two, not at all. But is more, although management/customers/end users will expect programmers to excel in this… well is not quite like this… 

Programming – Impact

This sub-dimension is to make the programmer aware of how the functionality she/he implemented will impact the final user. How many programmers think that because of their work an end user can be fired? 

■ Testing

Here I am thinking about Context Driven Testing(1) school of thought, especially Rapid Software Testing(2). This testing has its roots in cognitive psychology and epistemology(3). Please read the work of James Bach, Michael Bolton, Cem Kaner, Jerry Weinberg. 

If you are a developer and want to ignore this dimension than I advise you to think better. You will lose the chance to be an even better professional. 

Testing – History

James Bach, some time ago, pointed me out about the book “Your Code as a Crime Scene: Use Forensic Techniques to Arrest Defects, Bottlenecks, and Bad Design in Your Programs”. I was thrilled to see how many things can be deduced just by looking in the source control history. I was able to see the weak points of code and where to concentrate the refactoring. It was there in front of my eyes the Paretto principle, helping me in prioritizing work, but also helping testers in guiding their test strategy.

Testing – Testability of the application 

We need to give the application control points so the testers, programmers can exercise more easily certain scenarios. For example replacing more easily components. If we can’t do this, then is a red flag. 

Testing – Risks

4 sub-dimensions listed here are from the wonderful paper “Risks Analysis Heuristics (for Digital Products)”(4): project factors, technology factors, specification factors, operational factors. Studying this paper made me realize that coding is not just about if, else, while… instructions. 

The tester(s) have a different mindset, which should be viewed as a complementary to the one we already have, as a developer. Developers look at the code with the assumption that is right, but we need testers to look at the code with the assumption that is wrong(they are negative, they focus on failure(3)).

■ Business

All the code should be driven by the business need. Use cases/activities should drive the code and it’s structure. How do you know that a duplication is not just a temporary one which in the end will no longer exist? How do you know that you are applying ok the Single Responsibility Principle? Well guided by the business, the actors. 

Business – Analysis

I remember we discussed the heuristic “Mary had a little  lamb”(5). This heuristic help identify ambiguity statements which are in written form. The secret is to concentrate on each word one by one and then in combinations.  Let’s take the word “Mary”. We concentrate on this word by posing questions: Why Mary? Why not Giovanni? Let’s take the word “had” from the sentence: Why it had? What happened that no longer has it? What about present, future? Will it have another one in the future?. 

I love this technique. A lot of programmers want and  expect fully written requirements without side effects. But is not possible because:

–  it is a lot of tacit knowledge;

– is one thing for people to see the data(let’s not forget that even with the highest concentration we partially see/scan stuff), one is to pay attention to it and a different thing to act upon that data. (6)

Most of the time we go on the supposition that we know what customers/end users want. But is it like that? What if the end user does not know? What if the end user is unaware of certain things, on which we might help, which are possible and they may want it?  Why to code a thing and put a lot of effort in abstractions, algorithms when … maybe … that is not the path to follow? 

Is time for a lot of programmers to grow up and not ask for things just because it will be easier for them. Or search for excuses not to do work. They will code certain business things. This means they are directly responsible to understand and make sure they understand those business things. Christopher Alexander put it very nicely: “Please  forgive me, I’m going  to be very direct and  blunt for a horrible second. It  could be thought that the technical  way in which you currently look at programming is almost as  if you were willing to be ‘guns for hire.’In other words, you  are the technicians. You know how to make the programs work.‘Tell us what to do,Daddy, and  we’ll do it.’That is the worm in the apple.”(7)

Business – Impact 

Here is about building products and delivering projects that make an impact, not just ship software.

There is a technique for doing this which is called Impact Mapping(8). 

Impact mapping is a planning technique which uses in a very inspired way guiding mind maps in helping achieve this by guiding/facilitating the process. It shifts the focus from just doing what the customer orders to  focus on collaborating with all involved parties. The goal of the impact maps focuses on the question “Why are we doing this?” and gives a strong guidance on how to do this. This work collaboration between senior technical and business people is a must because the code changes based on this interaction. The abstractions, objects, architecture is strongly influenced by this. The business must be known by senior programmer – is a sine qua non condition for the seniority of the programmer. 

Business – Demo

At the end people will see the result of the code, not the code itself. This result is the desired effect. This means that the end user, the Product Owner have to see it and the programmer to get an early feedback. So this aspect can help the developer to be ok/prepared when doing the Demo. 

■ UX

Most often the work of the programmers is reflected in the UI. The final user does not see what is underneath. I really like how James Coplien makes the connection between object oriented programming and UX.  Is normal because via UX we try to translate as much as we can the mental model of the user and then that mental model must be reflected in code.

■ Human

Very often we forget the social aspect of programming. We do not realize that architecture, for example, has a social aspect. This is very important. 

Is not easy to integrate and make work two or more, different, personalities. I wrote before about this. There can be a conflict, a political stuff, sadness, ….All this can affect and will affect the quality of the code review.

In the mind map we put the word “Affinity” to be aware of all the subjectivity that can be present.

We put the word “Experience” because:

– of seeing bad examples. People who were considered seniors but they were affecting in a bad way how the review was done. Also the other team members, the juniors one, were influenced by these “seniors”;

– the experience plays a role in review because of the learning that can be done. Having an experienced developer that can share knowledge hands on;

– people with no programing experience , like some testers, were posing interesting questions. And the explanation towards them had to change a little bit;

About inattentional blindness, I wrote before about this aspect and code reviews.

Partial Conclusion

– Trygve Reenskaug had an interesting idea: the reviewer having the responsibility of the code being reviewed, not the programmer who wrote it. That for sure will change the dynamic of code reviews and it can make code review(s) more valuable/interesting not just a monoton approve step(9);

– After we (Alexandra Vasile, Marius Jantea, Ionuț Albescu and I) created the mind map and thought together at those dimensions and explanations I realized that in my subconsciousness I might have been influenced by the wonderful book named “Handbook of Technical Reviews” written by Jerry Weinberg(10);

– Seeing how code reviews are being done/treated gave me a clue regarding the dynamic of the team and possible sociological/management/organizational problems that might be there. I say this because a characteristic of complex systems is fractality;

– Do you feel to modify the mind map, to add more dimensions and sub-dimensions? Good;

On Part 3 of this blog post my friend Ionuț Albescu will continue in clarifying the other dimensions which were left out and also will give his personal touch on this subject of code reviews. 

(1) Context Driven Testing, http://context-driven-testing.com/ 

(2) Rapid Software Testing Methodology, https://www.satisfice.com/rapid-testing-methodology 

(3) James Bach, “Lessons Learned in Software Testing: A Context-Driven Approach”, https://www.amazon.com/Lessons-Learned-Software-Testing-Context-Driven/dp/0471081124# 

(4) James Bach, Michael Bolton, “Risks Analysis Heuristics (for Digital Products)”, http://www.developsense.com/resources/RiskHeuristics.pdf

(5) Jerry Weinberg, “Exploring Requirements” https://leanpub.com/b/requirements#bundle-page-exploringrequirementsone 

(6) Dave Snowden, “See-Attend-Act”, https://cognitive-edge.com/blog/see-attend-act1/ 

(7) Christopher Alexander, “The Origins of Pattern Theory: The Future of the Theory, and the Generation of a Living World”, https://pdfs.semanticscholar.org/7157/ee5a77c6fd64f8b8b2282225f113205370e9.pdf

(8) Gojko Adzic, “Impact mapping”, https://leanpub.com/impact-mapping 

(9) Trygve Reenskaug, “DCI: Re-thinking the foundations of object orientation and of programming”, https://vimeo.com/8235394

(10) Jerry Weinberg, “Handbook of Technical Reviews”, https://leanpub.com/handbookoftechnicalreviewsfourthedition 

The hidden dimensions of code review(s) – part 1

Context

The title image was created last year in December with the help of three friends (Alexandra Vasile, Marius Jantea and Ionuț Albescu) over lunch. Since then, I try to write about this subject. The trigger point was when, again, I saw a collocated team using exclusively a tool for code reviews and no personal discussion. It was clear to me that I was seeing some effects of bad things that happened there before and people were hiding themselves behind tools – is so easy to do this.

When I asked them why they did it in this way, they had a perfect excuse: audit. Smart things can be done so that an audit will not generate stupid behaviors. I saw that in our industry, at least here in my area, this audit is a good excuse for bad behaviors/recipes. I think there is a gross misinterpretation of the audit intention and on what should be done. 

So in this case, as in many others, it was about abnormal/bad work environment, lack of understanding subjects/topics  involving our day to day work(audits, code review(s), testing,… ). 

The good thing is that it forced me to think very seriously on code reviews and what it meant for my friends and I. I have to say that when  discussing with my friends, I had in mind the way how James Bach and Michael Bolton analyses certain things(1). 

Why is so important

I continuously saw how this subject, in my opinion, was badly treated. But why it irritated me so much? Well, among other things, because:

– this is how I learned a lot of programming stuff;

– it is so obvious the connection with testing and how helpful is to find bugs;

-it helped me see the connection with the idea of complexity(those things which are named boundaries and how the system will adapt/shape/modelate based on those) (2) ;

– the serious analysis which is being(should be) done can help us arrive at profound stuff;

Checklists

There was a time in my professional life when it was enough to say the words “checklist” and “tool”, when a problem popped out, and everything would have been OK in the eyes of certain managers. I think that those managers wanted simplicity, but most often we deal with an essential complexity. Somehow I felt that, for those managers, the situation awareness always could have been reduced to some simple/obvious things in a very fast way. Everything was reduced to recipes.

Context is everything. This means situations can have different dynamic. I am afraid that for those managers all the situations were all simple/obvious( in Cynefin terms)(3). So for them it was assuring that having that checklist would have acted as a guard with, almost, ad litteram actions described in that checklist – and it makes sense for the Obvious domain in Cynefin, but we are not always there, not at all. 

What if a checklist in complex domain, or liminal zone between complex and complicated, has a different form/significance? What if it means something else? I think that at this level we speak about compressed knowledge, cognitive activations, heuristics. They can’t be used ad litteram, is a lot of uncertainty. I understood this when I had the Rapid Software Testing course with James Bach. James had a list of  different words regarding testing on a slide, it was a huge slide full with words. Someone could have transformed that list in a checklist but…not ok. Each word from there could have been described in a lot of text and lots of actions dependent on context. 

This is how I realized that a professional expert might use a checklist in a different way than a junior would use it.

Maybe also with checklists, in these kind of contexts, the secret is … checklists “all the way down”(4) – and this is a whole different story, for sure not a simple one. James Bach said it so nicely: “Heuristics are often presented as a checklist of open-ended questions, suggestions, or guidewords. A heuristic checklist is not the same as a checklist of actions such as you might include as “steps to reproduce” in a bug report. It’s purpose is not to control your actions, but help you consider more possibilities and interesting aspects of the problem.”(5)

So, to come back to checklists as we know them which are used, most often as I saw it, for process compliance, just tick in the box – which for certain people, not professional who play good politics, can use to cover their ass :(. 

Pull requests

Instead of hearing the words “code review”, I often hear the words “pull request”. And of course the link of that “pull request” was sent by chat to the collocated colleagues. I would like to say that this is something new for me, but it is not. But what is even stranger, is those collocated people decide to do exclusively the code review via the tool. This reminds me of the team who was doing daily meetings via Skype although they were in the same office… 

I know there can be good scenarios to make code review via the tool. But to have it exclusively, no matter what, via the tool for collocated colleagues is rather problematic.  

The justification for pull request(s) most often is, again, the audit.

Why dimensions

I noticed that code review has only a narrow perspective in the eyes/actions of many people. Some aspects, intentionally or unintentionally, are not considered, like: the environment, the context of where that code lives, what generated that code, the possible unintended consequences of the code,… 

Is not enough to concentrate on those lines of code only. I say this because of 2 reasons:

– if something is not seen/observed does not mean it does not exist.

– I think that code has a complex aspect in it. This means that we have to see the connections also. Because in complexity is all about connections. 

The title image from this blog post was created to show the possible dimensions worth considering, at a certain moment, when doing a code review. It is a guideline.

Depending on the moment when code review is being done, certain dimensions(see the image below)  can be ignored while others will matter more. For example, in the image below, I want to concentrate on seeing how SOLID(6) principles were applied guided by the business need:

In the part 2 and part 3 of this series we (Ionut Albescu and I) will tackle these dimensions. 

(1) James Bach, Michael Bolton, ”A Transpection Session: Inputs and Expected Results”,  https://www.developsense.com/blog/2010/05/a-transpection-session-inputs-and-expected-results/ 

(2) Dave Snowden, “Boundaries”, https://cognitive-edge.com/blog/boundaries/  

(3) Dave Snowden, “Cynefin Framework Introduction”, https://cognitive-edge.com/videos/cynefin-framework-introduction/ 

(4)”Turtles all the way down – is an expression of the problem of infinite regress” , https://en.m.wikipedia.org/wiki/Turtles_all_the_way_down 

(5) James Bach, “Heuristic Risk-Based Testing”, http://www.testingeducation.org/course_notes/bach_james/cm_2002_rapidsoftwaretesting/rapidsoftwaretesting_13_heuristic_risk_based_testing.pdf 

(6) Robert C. Martin, “The Principles of OOD”, http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod 

Discussion about hiring developers

Context: A lot of managers, product owners try to solve the problem of hiring only based on some specific skill sets. This is problematic because it shows a control mentality. But what happens when the needed skill sets change? We modify the team based on the skillset or we let them the chance to adapt, learn?(1)  Which approach is more agile?

Shu: Hi Ri. There is a thing which puzzles me, regarding the process of hiring developers. Tell me if I am wrong. 

Ri: Hi Shu, which thing?

Shu: Very often I see an announcement with  the following form: “We need a senior .Net/React/Vue/Java/Angular/… developer”. For sure a certain language/framework matters if it is known, but  is not enough – at least not for a senior. Seniority should not be tied to a certain technology only. 

Ri: Is it maybe just an unfortunate choice of words? Maybe they want to say a “senior developer with Net/React/Vue/Java/Angular/… skills”

Shu: I do not think this, because of a strange effect I saw. Is about how teams are formed, at least what I have noticed in the past. 

Ri: Could you tell me more…

Shu: You have a well functioning team and a new project comes. That project needs a React developer. No one in the current team has worked with React, but they are willing to learn. But it does not matter. A well functioning team will be broken because of that. Is a framework which can be mastered in a week or two by a person who knows things. 

Ri: Tell me more about other skills a developer should have.

Shu: Design, analysis, algorithms, architecture, OOP, testing, experience with at least 2 languages and platforms,… 

Ri: Why at least 2 languages and platforms? 

Shu: Because the elders  say so? 🙂 

Ri: Why do they say this? 

Shu: Maybe because of biases. 

Ri: And also, maybe, because a developer who will not have fear approaching multiple languages/technologies she/he will understand what unites them actually. When doing this, in the end maybe, will understand that in order to adapt fast she/he will need a common ground. And this common ground transcends the language/technology. 

[Shu is thinking] 

Shu: By common ground you mean clean code, architecture, domain analysis, patterns, Liskov principle, algorithms,…. 

Ri: Yes. So, what you are trying to say is that the problem is about the locus of attention being in a slightly wrong direction? 

Shu: Yes…

Ri: There are some things, among others, to consider when hiring:

– Hiring is also about learning. We want those developers, testers to learn when the market/product/needs/conditions will change. We do not want to destroy circle and community of trust(2). “Experience is not a state but a process”(3) ;

– If the focus when forming a team is first on the technology used, then on the available people and only then on the social aspects of the persons in that team and the learning, then it will be problematic and sad. 

(1) Strongly influenced by James Coplien in Scrum – Product Owner class discussion [October 2019]

(2) Jeff Sutherland, James O. Coplien, “Circle of trust”, http://www.scrumbook.org/product-organization-pattern-language/development-team/circle-of-trust.html 

(3) James Coplien, Scrum – Product Owner class discussion [October 2019]

Scrum Master as a complex dynamical attractor

Context

Some years ago I noticed a strange thing when a new Scrum Master was assigned to a team. That team was already formed and just needed a new Scrum Master.  I observed that team before with another Scrum Master. I also observed the new Scrum Master with his former teams. My first reaction was: “How sad. This team will be transformed in selfish team, capable of doing bad things like sabotage. I saw the good part of these people, now I will see the bad part in them “. And indeed, it was like this.

Problem

I was starting to observe certain patterns of influence of people upon groups of other people, with interesting consequences.

At that time I was wondering how I can explain this in a way that it made sense, was coherent, professional and had some bases.

Why this

This was about a Scrum Master,  but I think it applies for Product Owners, managers, ….

This story is particularly important in context of Scrum. I say this because there is an important detail in the Scrum Guide, which is: “…Scrum (n): A framework within which people can address complex adaptive problems…” (1) (emphasis is mine). Here is about complexity theory, it’s deep stuff.

About the word “theory”

We, the IT, have at our disposition an entire body of knowledge from multiple disciplines which can help us a lot. Unfortunately this is, at least in my circles, rather ignored. And we base our ideas, work, actions on some “beliefs” or blindly.

This word “theory” is about science. Some sciences can be very useful in IT: cognitive science, anthropology, complex adaptive systems theory, etc.

We can use practices informed by theory (praxis)(2) in very practical ways in our day to day work. The fact that we know the theory can help us make sound statements/actions about novel and uncertain situations. We can do this because it is about validated knowledge from sciences(2);

In March, this year, I was at the Scrum training held by James Coplien. Everything he said there was not based on beliefs or impressions. He has done his homework regarding the scientific research. He knew how to back up everything of what he said by theory. I hope more Scrum people will follow his example.

This post is about complex adaptive systems theory. At that time, when I was observing the situation described in the Context section, I was looking at it through the lens of complexity theory, more precisely about attractors.

Atractors

Trajectories  that concentrate  on regular/normal patterns define the idea of system’s attractors. What is found in the basin of attraction will have its behaviours constrained/directed/channeled by the attractor, this means increasing the chance of going in one direction rather than in another. The behaviour of the system is constrained by the dynamics of the attractor. (3)

Can the attractors be persons? Of course, parental presence in a children’s party represent an attractor. Just imagine a children’s party with no parent(4). The presence of the parents there will influence what is acceptable and unacceptable behaviours.

Conclusion

● How a Scrum Master is chosen should not be based on frivolous reasons.  I say this because in complex systems, we deal with social phenomena, when developing projects/products, there will always be unintended consequences. Also in complex systems a small cause does not imply a small effect, not at all.

● Strange attractors are and will be present. The fact that the Scrum Master is a good or bad attractor or not attractor at all, will matter;

● I am so tired of seeing Scrum Masters who are chosen just because:

–  is enough that they know how to do reporting;

– the manager choosing the Scrum Master is enticed by the false appearance of words and behaviours of a certain person wanting to be Scrum Master;

– the person is a tester and is cheaper, in outsourcing, to make testers Scrum Masters and not developers;

– that person is very good in platitudes and reproducing the same things which can be put in just 4 A4 pages;

Scrum is not easy to do, is hard and that’s why I saw so many failures with it. I imagine this is because, maybe, lack of knowledge, fear, political games …of course is not a single root cause….

There is a lot more to say, of course. For example, other anomalies can be spotted in the situation described in the Context or in the points listed in how I saw Scrum Masters being chosen. But the intention of this post was to make the connection between 2 things: Scrum Masters and attractors.

(1) “Scrum Guide” , https://www.scrumguides.org/scrum-guide.html

(2) Dave Snowden,  “Of practical wisdom”,  http://cognitive-edge.com/blog/of-practical-wisdom

(3) Alicia Juarrero, “Dynamic in Action”,   https://www.amazon.com/Dynamics-Action-Intentional-Behavior-Complex/dp/0262600471

(4) Dave Snowden, “The landscape of management: Creating the context  for understanding social complexity”, https://www.researchgate.net/publication/228449006_The_landscape_of_management_Creating_the_context_for_understanding_social_complexity

Scrum Master – a view based on my experience as a developer

I am a little disappointed when I look around, and I see that many decisions that involve me, are taken by persons who will not handle the consequences directly, and all of these without asking me for an opinion. Is like I have to drive a car, but another person would position the seat the steering wheel,  and the rear view mirrors, without asking me if I reach to the pedals, if I am able to orientate myself in space by looking in the mirrors, etc.. and on top of that, that person would not even be in that car with me, when I will be on the road. I am not talking about the decisions related to what types of electrical wires to be used when the car was designed, or the size of the electrical fuses… but some small details that depend on the context.

A scrum master is a facilitator for an agile development team. There are multiple areas where there are facilitators, even though they are not called Scrum Masters. For example, in the automotive industry, one of the examples might be FMEA Coordinator.  

From my point of view, the technical expertise of the Scrum Master is a plus… it can be compensated by viewing the things applied to the context of the team but being technical or willing to understand the technical part would certainly help. My opinion can be influenced by the fact that I am a technical person. But here is why I consider this:

●  Having a Scrum Master with Technical Skills doesn’t mean that she/he should be involved in technical decisions, but it might be more comfortable knowing the fact that if needed, she/he can give an advice, or change her role if needed. On one hand it is an advantage to receive a verbal advice, but sometimes, a more involvement by the person who wants to help might do the difference – if that person knows and understands the context, perhaps with his/her experience might directly help in solving the encountered issue / difficulty. The way I see it is like traveling a lot on mountains, during the cold seasons, and having an emergency blanket.

This does not mean that you should use it every time. You can even decide that you won’t buy one, because you don’t see the benefit. But in a lifetime, there might be 1-2 situations where you could have used it (for you or for somebody else) and then having it on you might represent the difference between life and death. Of course, like having a Scrum Master with technical would not 100% solve all of your problems, same with the emergency blanket: It does matter to take it with you – it is not sufficient to buy it and leave it home when you are traveling, or to carry only the emergency blanked.

●  I practice ski touring. This allows a person, during the winter season, to follow paths on mountains, for example, uphill and downhill as well. In the safety equipment it is recommended to have with you, among other items, an avalanche transceiver. This is a small device that emits a beep sound on a certain frequency, and the signal becomes stronger as the person who is looking for you is getting closer. It is used, as the name implies, in finding avalanche victims. You carry it with you on emitting signal while you travel, and if you notice an avalanche, you might switch it on search mode, and try to find the victims as fast as you can.

I’ve told you about this, because I see a parallel in Scrum framework as well.

It is very good for a Scrum Master to know and understand her role very well, and all the needed steps described, but sometimes I think that these should be applied according to the context. For example, a transceiver would not be very useful if you are alone in the mountains and you are caught by an avalanche. So, if you are in a similar situation, you should pay attention to avoid as much as possible a disaster. What I mean is that you might be 100% covered by the book and definitions, if the context is not favorable for you, then you should adapt and not only rely on what you’ve read / experienced / known before. In a team context, applying for example Scrum Guide ad-literam, might bring frustrations and stress among the team members.

A scrum master with technical skills can cooperate easier with the product owner and expressing him/herself from the point of view of a scrum master, but also considering the technical part and intervene in order to prevent some things that can go wrong in that project.

The main purpose of a development team is to build a software product that meets its users’ expectations, in the minimum amount of time, and with very few or none defects – of course, this is the ideal solution. The main purpose is not to write automatic checks… or to decide to use story points when estimating stories / epics / tasks that are centralized in a tool; – these decisions should be taken considering the team’s opinions, and adjust them according to the needs, without abusing them, and of course to be applied on the context.

It is nice to have a sprint planning / retrospective / daily meeting (and useful sometimes), but not necessarily when you have a team of 1 person for example. Or even a team formed by 2-3 persons that can communicate and coordinate so that there is no need to have some of those meetings.

I have worked with Scrum Masters that were technical and Scrum Masters that were not technical and had positive and negative experiences with both. But, in my opinion, the issues were because some were operating by the letter of the law instead of operating in the spirit of the law:

● In most situations where the development team is too deeply involved in finding a solution for something that does not work, just talking with someone else would help a lot in unblocking the situation.

If we talk with a person who is not 100% involved in the development of the product, we might find alternative solutions, that as a team we somehow missed, guided by not looking at the problem from a critical point of view.

By not being involved 100% I mean that has some knowledge over the project, but it does not specifically work to develop / test it as a daily basis. That person could be for example, the Scrum Master. If she/he has some technical background, already knows the product, and some of the issues that we already encountered as the time passed by. 

● There are some parts of the management of a project that might be easier to be aligned, accomplished, and applied when the Scrum Master is a technical person, because she/he knows some things from both domains. She/he does not need to be an expert but can help a lot if she/he has some ideas.

A technical scrum master can cooperate easier with the product owner and expressing him/herself from the point of view of a scrum master, but also considering the technical part.

● A big minus is that by at least not trying to understand the process, to discuss it with the team members, and instead doing things by the book, can lead to uncooperative environment, and doing some things for the sake of being done. We are not robots, so we can think freely, depending on our previous experiences, of course. There is a saying: « beware of single source people who do not have an open mind. «   Ok, you may know exactly the words that describe a process, if you are not able to adjust it according to a context, then It might be better not to apply it at all or to ask for support.  

Perhaps this might be one of the scrum master’s role – to defer / avoid using things that do not make sense for the entire team. Maybe I’m wrong… but by not doing this, it might bring some frustrations among the team members, sometime this can reflect in management and in the overall result as well.

A flexibility can be part of the solution and apply things not blindly but depending on the context. In order of this to happen, a lot of experience is needed along with an open mind and courage to stand out of the crowd and work for the company / product / client / colleague’s benefit.

Rapid Software Testing course with James Bach

In November 2017 I attended the Rapid Software Testing course held by James Bach. Now, this course has been renamed as Rapid Software Testing Explored.

When I was on the train, going to the training, I imagined what I would have written about the course in those days. But I did not do that. After the training, I decided not to write about it and wait for a good period of time, to see how it will influence me in real life. I think this is the most important thing in the end.

I think that the time to write about this course has arrived. I hope you will find it useful.

Premises

There were some things preceding the training which, I believe, are very important and worth mentioning:

– I knew about the work of James Bach and Michael Bolton. Before going to the training I have fully read their entire blogs, and also the book “Lessons Learned in Software Testing: A Context-Driven Approach” written by James Bach;

– I knew about the work of Jerry Weinberg and have read a dozen of his books, especially the book “Perfect Software: And Other Illusions about Testing” ;

– For 2 years, before going to this training, I tried to understand the testing envisioned by Jerry Weinberg, James Bach, Michael Bolton, Cem Kaner;

– I was a software developer and team lead;

– Going to this training involved also  personal money/cost contribution. I mention this because I am strongly influenced here by the “skin in the game” concept of Nicholas Taleb;

– I have read the rst.pdf file, which was the course support material..twice;

– I have searched the internet for testimonials regarding this course with enough details, but I did not found a lot of them.

Feelings

I remember the strange feeling I had when I entered the class room. James Bach was there at his desk, but I did not dare to say anything. I can say that I was overwhelmed. It was unreal for me to be there.

After the first 2 hours of course, I realized that I knew so little about testing…

Details about course

I will try to write about this course via 3 dimensions:

-structure: the flow of the course in those 3 days;

-key words: it’s interesting to describe a training via some keywords used in the course, which can be enumerated coherently in a minute or so. I think it can have a good impact;

-pearls of wisdom: these are insights from the training, words used by James Bach. You will see that each of the texts is within quotes and preceded and followed by “….”. This is because the text has a context(story, experience, people, time, history, feelings, exercise, place). Some of those phrases took even an hour or 30 minutes to be described by James Bach for us. So there is a risk, for the reader, not to decipher the real meaning/lesson/message, but I put them there in the hope that it will convince/entice you to go at his, and Michael’s, training and find out more;

Structure

The course was centered around exercises, experiments and interesting surprises. These exercises were not easy at all, it was hard stuff(even if at the beginning, sometimes, for us it felt easy, we found out that it needs to be analysed first and we needed to ask for more information). I saw damn good testers being tricked. But James warned us.

Since these exercises and challenges were hard, he spoke to us about mistakes, about his mistakes. He told us that we should not be scared of them and that we should learn from them.

These exercises were inspired from real life difficult situations that James Bach experienced.

Each exercise was followed by a lesson/message. This was the beauty. If someone would have told me in advance the same lesson, I would have not understood it as I should – I am sure of that 100%. But you know why? Because we struggled with it, because it had put us in a delicate situation…try to imagine social pressure or a hostile environment, it is not enough to read about it, you have to live it.

The explanations were full of metaphors, so the message can be easily understood.

The interaction with James was via the Socratic method.

James intention was to make us learn more rapidly the things he learned in a much slower way.

Although I have read in advance the pdf which  accompanied the course, I realized that I have missed A LOT of insights…

Key words

Initially, I was overwhelmed by all the details from the course and I did not knew how to describe it in a short and coherent way. But, shortly after, I realized that I can use some words used in the training. So here they are:

context based risk analysis, compatibility regression testing, sanity check, test strategy, business risk, normalization analysis(from therapy), leader and its attitude toward tester, thinking outside the box, parenting, biases, process of thinking, pattern(s) recognition, efficiency and relative efficiency, sense making, common sense questioning, interview for hiring, diminishing returns principle, fallacies, design tests, cope with complexity and uncertainty, scepticism, risk mitigation, quality, agile, think negatively, tacit test procedures, normalized test sessions, cognitive science, social dynamics, psychology, social skills, qualitative methods, social pressure, functional blindness, hostile environment, data types bugs, boundary testing, heuristics, perimeter assumptions, defocusing technique, factoring, randomness, heuristic test strategy model, professionalism in software testing, hazop, inattentional blindness, audit testing, penetration testing, anchoring bias, mental testing models, adversarial images(from AI) , statistical correlation, product coverage outline, oracles, lifeboat argument, explainability/confusion oracle, person whose opinion matters oracle, defocusing, diffusion limited aggregation, deep testing, exploratory and scripted testing.

Pearl’s of wisdom

As I have already said, it is important to mention that these phrases do not have context around them. This is intentional. It is an invitation, if you like or find it problematic or… to dig more, even to go to the course. So:

-“… Sanity checking means shallow test coverage not deep test coverage. For the purposes of identifying insanities. You are looking for, not just something that is a little bit wrong, you are looking for something that is totally broken. If it’s totally broken I want to know about it …”

-“… All testing problems have a context, and if you know that context you will be able to make a sensible decision about what to do next. And if you don’t know the critical context variables then you will suggest a ridiculous answer …”

-“…. Rapid Software Testing isn’t rapid because you are typing faster than other people. It’s about thinking: Do I really need to do this? Why do I need to do this? What things can I set aside? And what things do I really need to do?  …”

-“… What a good leader should say, knowing how delicate and emotional testing is: I want to know if there is a problem. Don’t worry about the ship date. We will slip it if you say we need to. That is not your concern. What your concern should be: Can I find problems in this?. Let me worry about …”

-“… When you teach a tester how to test you are teaching that person to see something, to see things that you have never seen before …”

-“… Testing is an exploration and experimentation  process …”

-“… A paradox of RST is that: RST tells you that you should do the right thing. Don’t do anything unnecessary, that will save you time. But how do you know what the right thing is? First you’re gonna have to do wrong things to find out what the right thing is…So the paradox is that in order to get rapid you usually have to start slow to climb the learning curve…”

-“… Testing is an intellectual activity …”

-“… Never say:I tested it and it works. That is always an untrue statement. The truth is you tested it and you haven’t seen it fail yet…I tested it and I know it can work… “

-“…Testing is not about assessing that the product is good enough, it’s about finding problems ….”

-“…Counting tests is pointless ….”

-“…Rapid Software Testing methodology which is a mindset and skillset, but it is not a set of rules, it’s not a set of templates….”

-“… New novice testers don’t need lists. They need experience. How do you give them experience? Sit with them down and test with them …”

-“… Here is what you never do: you never derive tests from requirements, just don’t say that…”

-“… It is not possible to write down a test …”

How it helped me

This course helped me realize/do a lot of things:

– I will do everything I can to have trainings with the best in the industry. With those people who create concepts. If this means paying from my own money, so be it. I prefer to wait a year or two, but to learn from the best out there;

– Online trainings are important and have their place, but they can never ever substitute a live training held by one of the top of the top of trainers;

– At least once per year I would do the Rapid Software Testing Applied training;

– As a programmer, this professional testing dimension influenced me a lot. I can no longer define myself as only a software developer and ignore the tester part in me;

– I understood how and why so much money is lost in IT with superficial testing and the “automation” trend which now is being marketed ;

– I understood how to be a better mentor;

– It helped me by giving me force to write about testing and to defend it;

– I realized that, now, in IT, a lot of concepts currently used are not understood: testing, agile, scrum, OOP, architecture…

– I was able to help/coordonate a student, an experienced developer, for a masters degree in IT regarding testing. It was the first time such a subject was presented in university and it really impressed. I was glad that, finally, the real face of testing was presented in an institution which prepares students for the IT industry. I hope that in the future I will be able to teach a course on testing, but I am not yet prepared, I need help. I think that if testing, the real one, would be taught in University it would help even more the spreading of what real testing is, which in the end will help the industry as well as the people in it.

Conclusion

I liked this training a lot. I never would have thought that it would influence me so much.

How testers can provide value for developers

On the Rapid Software Testing slack Channel(1), one month ago, an interesting  question was put by David Högberg:

“Here is an interesting story and question from my friend in another Slack-group:

(translated from Swedish)

”Today was the last day of a sprint in one of the teams I coach at….

A backend developer has been very productive this sprint, he has pulled in more tickets than planned and finished those. Instead of pulling in another ticket on the last day he took the initiative to work on the unit test coverage and integration test coverage in our most important repository.

Now we have 100% code coverage for both unit and integration (API) and a branch coverage above 90%.

Besides this he always tests his own code, with both wide and deep understanding of the requirements. He always ask for feedback regarding his own testing, wants critical code reviews and practises risk-based exploratory testing.

This is what QA-coaches and testers will have to work with in the future. How do we create value for these future developers?”

….What are your thoughts regarding my friend’s question? How can testers provide value to developers who deeply care about code quality and at the same time are good at testing themselves?” (2)

The point raised by David Högberg is important and should not be ignored. Also, I saw that some testers were somehow speechless regarding this, like there was nothing to say/do/challenge/help.

There are lots of things testers can do. Testers must/can challenge things and there is a lot to challenge. So, for example:

1. The fact that what the developer does are unit checks , integration checks not tests.(3)

2. He said that he reached 100% code coverage for unit checks and integration checks. Actually code coverage is a measure and number for programers only. This is because it helps the developer to see areas not covered at all, not how well that area of code is covered. So, this number should not be exposed to management, because you can have 100% code coverage without code actually being covered. From the moment when you hear that code coverage number is exposed, there might be trouble – I know it sounds strange, but for me is a sign of exaggerated confidence. Maybe I am wrong, but for me is a heuristic for looking with care, because of the things which can’t be articulated in simple boolean confirmation statements.

I have to say that each time when I discuss with managers I try to do my best to convince them to ignore the code coverage number and the fact that it means nothing.

By the way, when I have the occasion I ask programers this simple question: So you are so confident with the unit checks?  If the answer yes, then I ask: Good, are you ready to bet you salary for 3 months that I’ll not find any problems? Usually, I perceive hesitation.

A good technique to verify the actual lines/conditions, in terms of coverage, is mutation checking(4). Even with this technique, I have in my mind the difference between testing and checking! (3)

3. As a tester you can help the developer with the sampling, the data she/he is choosing. And, as you know, this is not a skill which can be learned in a day or two. Sampling at the unit and at the integration level is important and I saw a lot of times unit checks and integration checks which repeat themselves.

I do not mention that testers, need to know very well about cognitive biases, critical thinking.

4. The developer said that he/she made unit checks? Perfect. Challenge that. Take a look at this brilliant article by James Coplien: Why Most Unit Testing is Waste . You’ll notice that this article is not so easy to digest, is deep – it speaks about the theory of information, also it does not follow the current dominant thinking about unit tests. I’m sure it will help you to make that developer think. Maybe negative reactions can take place, but is OK, we want fruitful disagreements.

5. The developer said he/she is doing TDD? TDD does not help at big architecture and OOP. It helps, maybe, at the micro level, but for sure not macro. This means architecture, OOP is fully exposed in a big and nasty ways. Maybe integration checks can help to see that the parts are fitting together, but are not helping to see that they are ok built. What this means? Well… nasty things are hidden at the integration boundaries. Developers don’t do OOP, but Class Oriented Programming. OOP would mean to reflect the mental model of users, which this does not happen. All the work is split between a lot of classes and functions. I think you can imagine the dangers.

Regarding TDD, I noticed that some persons begun to make some research about it and it is interesting: A Comparative Case Study on the Impact of Test-Driven Development on Program Design and Test Coverage or Does Test-Driven Development Improve the Program Code? Alarming Results from a Comparative Case Study

By the way, I also liked this post: TDD is dead. Long live testing

So, please, go and see the research, I doubt a lot of programmers do that…

6. Unit checks and integration checks is also code, this means that is prone affected by things like: bugs, maintenance, design, analysis, architecture.

7. I love the article of Max Boisot with the patterns(5). For example: for 10 dots there are 45 possible links and 3.5 trillion possible patterns. I’m sure in applications there are more than 10 connected dots. Is not possible for a developer, no matter how smart he/she is, to be sure he/she covered all the possible patterns and also to be aware which pattern is more important than other(s).

8. Regarding “A backend developer has been very productive this sprint, he has pulled in more tickets than planned and finished those.” Maybe I’m interpreting this in a wrong way, but sorry… I’m not interested in the productivity of the developer. This is a very nasty stuff. For me is important that the team and the whole work to be ok. When I read productivity of individual, I begun to doubt the good functioning of the team and this is bloody dangerous. Is like measuring tester for the bugs introduced, sorry, but no. Were there any other members actually working on stuff? If yes, why he did not took care to help finish that?  – I’m deducing this from the text, maybe I’m wrong. In Scrum the team works on a single PBI at a time.

Still the list is short. I would have added about cognitive dissonance, of how the team structure affects the code structure, but you got the idea…

Conclusion: So how testers might create value for these kind of developers? I think that when a tester will put this question then is a good sign because learning/helping continues.

(1) Michael Bolton, “RST Slack Channel”, https://www.developsense.com/blog/2017/10/rst-slack-channel/

(2) Text used with permission, thx David Högberg

(3) James Bach, Michael Bolton,”Testing and Checking Refined”, https://www.satisfice.com/blog/archives/856

(4) Robert C. Martin, “Mutation Testing”, https://blog.cleancoder.com/uncle-bob/2016/06/10/MutationTesting.html

(5) “The problem of connecting the dots”, https://sensing-ontologies.com/the-problem-of-connecting-the-dots/

Discussion on caring about teams and people

At the apparent level it seems it’s a vague discussion, but it is about Shu’s internal fight, on how to choose between people. People that are so important to Shu. And actually, even with all scenarios, Shu will lose on some level.

Ri: What is happening?

Shu: <<a sight>> I am not good in having a team.

Ri: In having a team?

Shu: Yes. You know..as Scrum master or ….whatever…. You choose the title…

Ri: Why?

Shu: It hurts a lot when the members of the team are no longer near me.

Ri: Hmmm. Why do you say that?

Shu: I can’t handle it when I loose a team, or members of a team. It hurts like hell. If I would know they are ok, then it would be ok – it’s hurting only because of my ,let’s call it, “selfishness” :(.

Ri: For those who are ok, you can’t and you shouldn’t  do anything… just to accept it…,but it’s ok to miss them.

Shu: Yes

Ri: And for the other ones, what are you going to do about them?

Shu: You know…if I wouldn’t have known that you care about me, I would have told you to f..k off with that question.

Ri: Hmm. I suppose you’ve heard it asked in pervert ways too many times.

Shu: Yes. Along with a lot of other questions/remarks…

Ri: So….?

Shu: I feel I should fight for them. But I will most probably lose.

Ri: Will it hurt?

Shu: Yes, it will hurt like hell. You cannot imagine, because there are other people, dear to me, who need me and I care about them a lot too.



Ri: Do you think you can do something which is correct also for the others and for the team, but not for you?

Shu: Yes, but I think, it will hurt me really bad and I will lose.

Ri: Hmmm. Then, I think, you know what you have to do. But you’ll suffer.

Shu: Hmmm. Yes.

Ri: Good then :)), at least you’ll have something to share, when you will be older. But you forgot something…

Shu: What?

Ri: What if you would have chosen the other people, who need you, and the result wouldn’t have been ok either? And by ok I mean that it wouldn’t turn out the way you desired it to?

Shu: <<is sad…he/she did not think about this>>

Ri: It seems it’s not about the fact that it’s easy to have a whatever team. It’s about the fact that it’s a big deal to have a team, a real one.

A discussion about code, data structure, objects

Shu: I need your opinion on something related to code. Maybe the other person, with which I discussed,  is right. But still something I feel is not ok.

Ri: Ok. Tell me more.

Shu: I need to pass to the UI a list of address types. For example address type of home, or address type of work,… My initial idea was to send a simple collection with key value pairs. Something like:

           new Dictionary<int, string>{{ 1, “Work” }, { 2, “Home” }}

           or

           new List<AddressType>{AddressType.Home, AddressType.Work}

           where AddressType is an enum type

And then the UI would know what to do.

Ri: Ok

Shu: My problem is that at a code review I was being told that I should return “a list of objects”. And each “object” should contain the same 2 data. Something like:

           List<AddresTypeData>{new AddresTypeData{Id= 1, Name=”Work”}, new AddresTypeData{Id=2, Name=”Home”}}

Why the latter is better than the first?

Ri: It’s not. Conceptually are the same thing. Ok, we can speak of some advantages regarding reading the code, or what happens at runtime. But, I think that something else needs to be clarified tough.

Shu: What?

Ri: The reviewer said “object”?

Shu: Yes

RI: What he proposed is an object?

Shu: Yes

Ri: Look …<<a sigh>>… is a big difference between a data structure and an object. Maybe what he wanted to say is that he wanted to represent the idea of Address type in a different data structure, this is a different thing.

Shu: I do not understand what you are trying to explain.

Ri: Look, objects are like biologic cells(1). Cells which communicates between them to do something. Those cells have a sense in an ensemble of connecting cells, ensemble which tries to do a certain thing. What cell send to another cell is a message. (For example, for neuro cells from the brain that message is an electrical signal. This electrical signal jumps from a cell to another, is not continuous. This message flows via synapses. ). Those messages can have their own structure, but there are still messages. A cell will not do the work of another cell, and the cell will know to whom to send the message. An object you’ll see it truly only in the connection with the other objects, which as a whole try to solve something.

Shu: What do you want to say?

Ri: Semantics matters, and for me it was a sign of confusion of what objects and OOP mean. And I wanted to underline this. You were speaking about data structure of a message not objects, which are a different thing.

(1) Alan Kay, “Dr. Alan Kay  on the meaning of “object-oriented programming””, https://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay_oop_de