|
Software configuration management is fundamental to top-flight process
July 27, 2006
URL:http://drdobbs.com/architecture-and-design/software-configuration-management/191204703
Pablo is a SCM consultant and software developer currently working for Codice Software, in Spain. He can be reached at [email protected].
Software configuration management (SCM) is all about organizing, controlling, and managing change in software as it evolves. As a discipline, SCM defines a process for change control. Unfortunately, SCM is often thought of as only necessary for really big software shops undertaking really big development projects. I say "unfortunately" because, in practice, the principles of SCM can--and should--be implemented in software projects of all sizes.
For instance, a couple of years ago I moved from a large-scale, distributed development team making software for digital TVs, to a small group responsible of the maintenance of an ERP system. With not much more than a compiler, I found myself responsible for a 400,000 lines-of-code Delphi 5 project that was the code base for five different products. In a perfect world I could have hired some large group of software engineers, bought management, estimation, version-control, defect and task-tracking, and automated testing tools. Unfortunately I didn't have resources, so I had to be careful in my selection. If there's one thing I've learned in the programming trenches, it is that you can survive with very few tools--a compiler, editor, and strong version-control system. Why version contro? Because you can take advantage of it to not only put your files under control, but also manage the efforts of the whole team.
So I pushed management and ended up getting Rational Clearcase LT, an entry-level SCM tool from IBM.
At first glance, it seems SCM is a big shop toy. Why bother with versions, releases, branches, and the like, if there are only four developers on your team? Doesn't that bring more overhead than benefits? The answers are: "yes," "you need it," and "no, it doesn't." Even small size teams can produce really good software if you can count on the right professionals, use the right development technologies, and organize your efforts correctly.
The main difference from a SCM point of view between a four-developer and 100-developer group is that the smaller group produces less code in the same time. But the quality must be the same, and the efforts have to be even better handled, because the resources are scarce.
In a small shop you can't afford losing some code because it was "unexpectedly overwritten"--your team has better things to do than rewriting it.
Before putting the SCM tool to work, I tried to order the source tree, coming up with something like Figure 1.
Figure 1: Project's directory structure.
What was placed under SCM was already in the source tree--lots of directories with tons of files like .pas and .dfm together with some .dpr. I modified each project so that their binaries would be placed under the bin subdirectory, and the .dcu would be located in obj, so that we were able to keep the tree structure clean of intermediary files.
Our product had been under development for a long time, evolving through versions 1 to 3, so at outset we used Delphi 1. By the time I joined the company, we were compiling with Delphi 5. Since many of the .dfm files were in binary format, I decided it would be a good idea to convert them all to text format so that the SCM tool can keep track the differences correctly.
Next I tried was detecting all the dependencies that the projects had on third-party installed components, even in-house ones. I wanted the source tree to be as independent as possible from installation issues, so moving the whole src tree (and I was thinking on workspaces) wouldn't be a concern. So I placed all the .dcu from the different components (each group on its own subdirectory) under lib, and updated the projects directory configurations so they used these .dcu to compile.
In doing so, I implemented the following policy: The bpl would be copied to a bpl subdirectory under Delphi installation directory, and installed in the IDE from there (the same for every developer in whatever machine they are). The .dcu files were not an issue anymore, because they were located using relative paths. This approach not only introduced a way to handle compilation, it also provided a way to deal with external elements: Whatever can be precompiled will be used in its binary form. We ended up with a tree like Figure 2.
Alas I forgot (and still regret) to reformat the code. Before creating the repository on a SCM tool, you are handling the prehistory of your system. Whatever happened there remains obscure. Once it is under source-code control, every change is controlled. Minimizing changes is good to keep a consistent view of modifications. I you need to fix a bug which involves modifying one line but you reformat the whole file, it won't be clear what you needed to change afterwards. This will be quite confusing. Unfortunately the same happens with a controlled reformatting, the code is better, but it is almost impossible to trace back changes easily. Consequently, reformat the code before or make the reformat just after the initial code import into the SCM tool.
To make the initial code import, I first created two VOBs (versioned object base, the project repository)--one for the products and another one for the components. (Again, we were using Rational Clearcase LT, and the terminology varies from tool to tool.) Then I imported both all the code for the project and for the different components.
Once everything was there, I compiled each of the applications (having a project group or makefile-based build system helps). If it goes well, then you have your first controlled release. So what I did was label the entire source tree with the name of the release, V151 in this case.
Once the code was there, we prepared some documents about setting up views and filling in the configspecs.
Putting all the codebase under source control was the first step to bring some order into the project. But then it was necessary to introduce a whole working process together with the tools.
Our process was quite simple: Every single change to the source code had to be registered into a homemade task control application. It would assign a number to the issue (being a bug, new feature, or whatever), and with this number (and in our case a prefix) we would create a branch in Clearcase. We are using a branch per task pattern, so every single development is isolated in its own branch. It would be a big burden for several SCM tools, but not for clearcase.
So each task has its own branch and is assigned to one developer. Hence each developer gets completely isolated from the others, his own changes won't affect his colleagues, and neither he will be bothered by incomplete modifications from the rest of the group. You will get this benefit using any SCM tool, is just a consequence of workspaces, but if you add branches to the picture then developers get extra facilities like version control on their ongoing work. This is really useful once you get used to it.
Figure 3 shows the development cycle we used.
Basically the project manager is responsible to introduce tasks in the in-house task control tool. Tasks are assigned to developers. Each developer works on one task. Once a developer finishes his work on a task, he undertakes unit tests. The task won't be finished until all the unit tests are passed. When we started with SCM we didn't have unit tests in place, but as we were progressing we started to introduced this technique, which proved to be helpful. We use Dunit. Once a task is finished the developer moves to another one, creating a new branch and so on.
After some time, you end up with a pool of finished tasks, with their associated branches containing all the modifications.
Since we are dealing with a business application, it is sometimes hard for developers to really know the implications on the rest of the program. Consequently, we introduced the following policy: Every single business-related task has to be validated by a business expert, a consultant in our case. So to finish a task the developer has to complete the following steps:
Periodically a selected group of consultants check the finished tasks in the tool, run the related executables to determine whether the modifications are correct, and if they agree on the changes, they mark the task as verified in the task control tool. Once there are several verified tasks, the tasks are integrated to create a new release. There are several possibilities about integration periodicity. Some developers argue in favour of daily integration, others for integrating continuously. We decided to integrate once a week. The period depends on the amount of work your team is able to produce. If your team is big enough to complete a large amount of tasks every day, then integrate once a day. In my case I need a week to have a good set of verified tasks, so we create a new release every week.
A more agile approach says that a continuous integration is better. In my opinion it depends on the nature of the software you're developing. If stability is your biggest concern (and it is in my case), then I prefer to have somebody controlling every change that is made to the software. If velocity is the main concern, then is better if every member on the team can integrate its own changes. Also having somebody responsible of the integration process hides the complexity of the task to the rest of the team, than won't be bothered with the internals of the SCM tool.
Once a new release is created, the source code is labelled and a new baseline is created. Recently we have created a test group integrated by both consultants and developers. The group has created a big set of automated tests under Rational Test Suite that checks most of the functionalities of the application. Nowadays we are not yet covering all the functionality, but we're pushing to have a quite complete test suite. So before labelling a release we run the automated test suite (now it takes about 12 hours), and if it passes then we label the release.
Once the release is labelled, the ongoing tasks are asked to be updated to the new release so that developers work against the most up-to-date release.
Choosing a certain periodicity for integrating has some consequences in planning. Since we have chosen to integrate once a week, ideally the biggest task a developer works on is a week long. But we always have tasks that need more time than a week to be finished.
Having a task that lasts for more than one release is not a problem, because you can always rebase your work, updating it with the latest changes from the release. But is clear that doing so introduces some overhead and complexity in the process. It is better if you can avoid it. Figure 4 shows a task that is not completed in one release period and needs to be updated when the release is created. So if possible would be better to split long tasks so that they can be completed inside the release period.
Again, during the finalization of a task the developer has to add version information to his executable. I started doing that modifying the version properties of the project with Delphi. The problem is that using an SCM tool you have to check out all the project files (not only the dpr) to modify and save the version information. So I decided to create a separate version file and include it as a resource (version information is a given resource). So during compilation a .res is created with brcc tool from the .rc file, and version information is added to the executable. Figure 5 shows the contents of the .rc file.
1 VERSIONINFO
FILEVERSION 4,3,211,0
PRODUCTVERSION 4,3,211,0
FILEOS 0X4
FILEFLAGS 0X04
FILETYPE 0X1
{
BLOCK "StringFileInfo"
{
BLOCK "0C0A04F4"
BEGIN
VALUE" CompanyName", "CIM, s.1."
VALUE "FileDescription", "V4.3"
VALUE "FileVersion", "4.3.211.0\255"
VALUE "InternalName", "Xmas\255"
VALUE "LegalCopyright", "Copyright \251 1993-2004."
VALUE "LegalTrademarks", "marca registrada de\255"
VALUE "ProductName", "BL211"
VALUE "Fecha", "16/12/2005\255"
VALUE "OriginalFilename", "exe\256"
END
}
BLOCK "VarFileInfo"
{
VALUE" Translation", 0x0C0A 0x04E4
}
}
To compile the .res resource file I created a makefile, and I ended up using the makefile to compile both the finished tasks and the releases. So we compile under Delphi for normal development, but as soon as we finish a task we go to the build subdirectory and type something like: make TESTUNIT to run tests or buildtask to execute an script that will extract the task name using clearcase commands, create a .rc version information file with the task information and build our executable with that information.
Using a makefile is probably not the easiest or more fancy way to work, but we can easily create executable without debug information, run unittests or compile and include certain resources (now we include SQL sentences for different databases as resources, and an internal manager class loads the appropriate ones depending on the database you're connected to). It was specially helpful to create releases without debug information at the beginning, when we had five different Delphi projects (now we have only one dpr). Otherwise we would have to open each one of them, uncheck the debug information boxes, and compile.
Introducing a powerful SCM tool in a project is one of the main steps to follow if you want to increase the quality of your process. Activities will be better managed using such a tool. Finding bugs is easier because you can trace back any change done to your code base. And finally, controlling releases in a formal way introduces a work discipline that is helpful for the whole team.
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.