DevOps part 5

Before starting posting about DevOps I had a small idea about what it was. At my work we use agile development process (Scrum) to establish some goals and all of them are shared through all stages of development. Since my area is testing, it is simple for me to work as a team and follow the DevOps (kind of) process among all areas. At my job we use a private platform in which we share everything that has to be with the software.

We use some types of queries to assign and follow up the tasks, and internally each team has its own metrics and goals. Since I’m in the testing area, what my team gets is a list of issues, or sometimes we need to find them, and then we assign the issue to a developer and follow up the fixing process.

Some advantages of using this system are:

• Intercommunication of all areas.
• The process is transparent to all employees.
• Queries let you open new tasks at all time.
• Each task is assigned to a group. (one person per area)
• The managers have access and control at all time.

The disadvantages that I see are:

• The system is sometimes hard to understand.
• The version control is visible but sometimes hard to work with the same version.
• New members have a hard time learning it.

Now, excise tasks is a concept related to DevOps. These tasks are the ones that take most of the work for developers. DevOps can reduce the amount of time that employees take in these kind of tasks, since everything is well distributed and organization time could be less.

Blogging is a lifestyle

I want to make this blog based on Ana’s post of Blogging and me. As she writes on her post, there was a time that I felt that blogging was difficult. And I recommend reading her blog and then coming back here.

I won’t lie and say that I do all these blog posts because I love blogging, they are actually assignments of a class called Software Quality and Testing. However, I do like expressing myself in things that I learn daily.

As Ana explains on her blog, starting with this is not an easy task. People around the globe are watching what you are doing, and may have a good or bad point of view. But I don’t really see this as a bad thing. There are always people that like what you do and how you express, and people that don’t.

When talking about expressing yourself in general, it is very important to do it based on something you really like. It doesn’t matter if the topic is not of your knowledge, but to feel something about it. If you begin doing stuff based on a trendy topic, you will get tired pretty soon.

Retrieved from https://www.bhf.org.uk/informationsupport/heart-matters-magazine/wellbeing/how-to-set-goals

I have to say that I don’t have enough time to blog, but I would like to. Doing it is something that I have always on my mind. An d even though these blogs are just for a class, I enjoy making them. My advise is to do what you like, and as I did my whole life, don’t let anyone to tell you what to do or that you are terrible doing what you like. Those haters will only stop you, and believe me when I tell you that ignoring them actually makes you stronger and wiser.

 

Python Testing

This is a screenshot of the annotation using Hypothes.is

From the same web page I searched how did Coca-Cola’s web page look in 1997.

Then here is a screenshot of PyCharm usage with pyunit.

I loved the way the Unit Testing and TDD in python course was driven. The presenter showed how testing should be done in reality and explained it in a clear way and simple steps. It was easy to follow and even a person that don’s know about programing at all would be able to get the job done.

I didn’t like the fact that it is too slow. The videos are ok, but the course in general starts to slow. However, I believe that this is necessary sometimes.

Building up the DevOps ecosystem | Part 3

For the next part I’m going to configure my Github so it can be more secure than before. Since I am working all the time with Github and Bitbucket, it is important for me to have the best security as possible, so first I stablished a two-factor authentication process.

Two-factor authentication enabled

Then I set up my SSH keys to connect my Github repositories using ssh.

SSH keys of my Github account

These two steps were really easy to perform and can be done just by following the Github documentation.

Now with all of this I can clone my repositories with ssh (something that I already do) and use my keys to connect with my Github account more securely.

Building up the DevOps ecosystem

For this task I will be focusing on building up a Linux Server Setup for later usage in DevOps. This could be taken as a tutorial, so feel free to follow post if you like it.

The first thing is to install the Linux OS. I will be using Ubuntu Desktop 18.04.2 LTS (you can download it here) and install it in Virtual Box. You can put it directly into your machine, although I don’t recommend it if you’re only going to use it once.

Installation of Ubuntu Desktop.

Now, let’s make sure that we have the proper environment for developing software. With this I’m referring to all what we need for our favorite programming language and also setting up git. In this case I will be using python. If I’m not wrong, Python is already installed in Ubuntu, but if not you can follow the instructions that appear when you type python –version. I will recommend using python3 so to use it automatically run the command alias python=python3. As Python, git is already installed in this version of Ubuntu.

Image showing python version.

Git command working.

Now it’s time to setup web deployment, and for this I’ll be choosing the LAMP stack. It is very simple to do it. You can follow the next post as I did, it is pretty straight forward.

Running PHP-FPM with Apache.

After that, lets  program Cron to pull a GitHub repository every day at 7am by adding a line to Crontab file.

Crontab file modification to pull repository every day.

DevOps: More than Software Development

DevOps is a a term that is used currently today that is associated with agile development. And it actually comes from the collision of two trends in the IT area: the “agile operations” and the relation between development and operations.

I truly believe that DevOps is a compound term, but not only in the spelling sense. For me, the definition of DevOps is a combination of development and operations working structure to perform better in the software lifecycle, by integrating techniques from both areas that could be well applied in each stage of software development.

DevOps has a strong relation with Agile and Lean processes, therefore it is always looking for improvement. In all the process, people from areas like engineers, system administrators, operation staff, DBAs (from Ops) and developers (from Dev) mix their abilities to improve the software quality (involving QA sometimes) and go beyond the Agile methodology.

Combination of Dev and Ops taken from: https://dzone.com/articles/the-rise-of-devops-engineers-in-the-current-market-1

Based on the blog post What is DevOps?, I agree that there are three primary practices that involve DevOps:

• Infrastructure automation: create your systems, OS configs, and app deployments as code.
• Continuous delivery: build, test, deploy your apps in a fast and automated manner.
• Site reliability engineering: operate your systems; monitoring and orchestration, sure, but also designing for operability in the first place.

And so, as I gave my opinion about DevOps in this post, there are many myths from people that say that DevOps is developers going over operations job, or that it is a new culture in the IT area; the real thing is that this movement goes way beyond and is more about integrating the development and system management areas, so that developers can focus mainly on coding and delivering in seconds. With this changes the software development could be more flexible, with better quality, cost efficient and faster.

Fix them all!!! The secret life of bugs

Even though the topic seems like a Walt Disney movie title let me assure you that this post goes in a totally different direction. However, as in those movies, in the case of bugs in software, each one has a story related to developers, testers, people who discovered it and solved it, among others. The paper in which this post is based on (The Secret Life of Bugs) uses these stories to identify common bug fixing coordination patterns and to give new tools in the software development.

There comes a time when modern software development gets a huge amount of bugs, and since this number is so big, productivity changes in meaning into amount of bugs fixed. All of these issues are stored in what we know as repositories, which become a storybook for all of them. But the paper goes way beyond and talks directly to the key actors to discover the patterns that are commonly used to fix bugs. The authors also relate their work to some researches that were made before.

The goal of the study is to provide an efficient and contextualized way for bug fixing tasks. So two research questions took the lead at the beginning: how is the process of bug fixing (lifecycle)? And, do electronic traces of interaction provide the necessary to understand the story of each bug?

To understand better the process of the study I will give a brief description of the two phases and then give my opinion.

Multiple-case

For this part, the authors selected a group of bugs that had specific concepts and behaviors like the status of “close” and followed the same methodology for all the cases. By obtaining the bugs database from the repositories they could get owners and know who modified the code. From there, they contacted the people who made changes and saw if they were relevant enough to ask about the history of the bug.

I found this process really complex, but at the same time is a good way to get all the story of the bug. They got pretty solid information and got it from first source reference.

Survey

For the survey the process was to interview some workers at Microsoft about their last project where they solved a major part of the bug. First the person gave them general data about the bug, then they spoke about the general patterns, and then see if they told the full story about the bug.

Even though this way is more personal, it could also be biased, since one person may not remember the full process and the story may have some errors.

…

At the end whichever process is taken, there are always missing things in the process of fixing bugs, so it is kind of impossible to find a direct path to fix them. But what I can tell is that human error is still better than machine error, since most of the surveys show less missing information than just following the track of the recommits in the repos.

For me is clear, that automation is sometimes helpful, but still is not as helpful as a human being. In the bugs’ lifecycle there is no 100% effectiveness, we can make some adjustments and try to follow some recommendations, but at the end preventing bugs is impossible, and try to follow the way of fix them too.

Back to the basics

Since I began programming I never though that testing could be an important part of the development process. Now that I’m close to graduate and I’m even working in the test area, I recognize that it is one of the most important parts. But, why do we continue careless about testing? Or is it just looseness?

I think that one of the reasons is that we just want to finish the program and expect that everything works fine, but certainly this never happens. I don’t really remember a program that was perfect since the beginning, and the person who tells you that these programs exist, is a liar.

For this post I will go to the first programming language that I learned (Python), and show you a simple way to start testing your software with a simple tool often called PyUnit.

The Python Unit Testing framework or PyUnit, is a Python version of the known JUnit for Java, and it’s very simple to implement a testing program just by importing the “unittest” framework. (You can follow the next steps below and even get my code from here)

After that we need to import the program that we want to test. For example, if our program is called hello.py, on our test program we will put “import hello”.

After importing both, we need to create some test cases, but in order to create them, we need to create a test class, inheriting from “unittest.TestCase”. The next step is just to add as many test cases as we want. This time, I’m making two: one for the sayHello method and the other for the add method.

In each test case, I will use a function called “assertEqual”. This function lets me compare the result of the hello.py program, to the one I desire. So if I do everything correctly, both tests should pass, otherwise a failure will be displayed.

Last, we need to add the next lines at the bottom of our code, in order to run the test directly from our console:

After that we just run our program as a normal Python one, with “python test_hello.py”

For the demo, I made my test fail, saying that 10 + 5 = 3 (which is false). And that will give us this failure:

This screen says that our test program ran two tests, the first one was successful (represented by the dot), but the second one failed (represented by the F) because 15 != 3.

If we actually run the test case correctly, then something like this will appear on your console.

So that’s how PyUnit works. As you can see, it is very simple to test our programs. I know that this one was very simple, but if we actually create a habit for testing software, then our documentation will be better, we won’t have to worry about failures and we will be certain that everything works fine in our software.

For more information about PyUnit go to this page.

And if you want to see the complete code, go to my repository at GitHub.

Something about testing…Or, should we change it?

At the moment of coding, we never think about a way to improve our software, unless this is your work, otherwise, we just concentrate on getting the work done and never test it.

Through my career I have learned many ways to manage my time when making a project. I will accept that I was one of the mentioned above, but fortunately I changed for better. Somehow, when your major is Computer Science, Software Engineering or something related to the IT area, you hear about Kent Beck.

Precisely, this blog is about Kent Beck, specifically a podcast  called “test && commit || revert”. In this podcast the announcer begins by giving some complains from developers about the Agile movement (being Kent one of the 17 signers of the Agile Manifesto), to which Kent answers that there are some crazy people that are reviving the Waterfall method, something that is not well recommended in today’s software development.

During the conversation Kent comments that the act of testing needs some courage and braveness, because we need to give some value to what we are coding. And argues that developing software means risking your work, which is FINE!

Picture from: https://www.pexels.com/photo/portrait-of-young-man-326559/

Kent hardly believes that good software ideas are not challenging enough. He prefers the bad ones because there is no competition and anyone can improve their software skills with what he calls the TCR process (the main topic of this podcast). Basically, this process consists on running some tests, then commit them and improve them. But these tests are closer to each other than what they appear, since Kent recommends programming in small pieces.

Sincerely I see this process as a fast and efficient way of coding. It lets you advance in your work with the ability of testing, and improve it in every iteration; something that really contrasts with making the full program and then suffer when testing. TCR pushes you to make changes, and that enforces your coding skills.

At the end TCR is just another workflow alternative, just like TDD, Limbo and test after. These alternatives should adapt to the programmers, and not the other way around. As Kent said, everyone is in charge of their own learning, so there’s no need of following a linear path in order to learn new mythologies. If the new ones, like TCR, work for you and could adapt to your recent project, then go ahead and dive into the knowledge. Otherwise, just keep making great programs and improve them with your actually skills.

Picture from: https://medium.freecodecamp.org/test-driven-development-what-it-is-and-what-it-is-not-41fa6bca02a2

At this moment of my life, TDD works fine for what I’m doing. It actually matches the way I work at my job. And shaving testing in mind, is not bad at all. It actually makes programming easier and fun to do. Just as a recommendation, go ahead and hear this podcast, it may give you some information that you never considered before.