Quicker Development and Publishing of Conda Packages
Submitted By Jenkins User Brendan Elmes
Manual testing led to longer build and upload cycles, security that could be improved, and frustrated developers. Automating testing was the way to go!
Organization: International Banking Organization
Programming Language: Python
Platform: Docker or Kubernetes, Linux, Windows
Version Control System: Bitbucket Server
Build Tool: Tox
Community Support: Jenkins.io websites & blogs
An international bank sees quicker development,
improved security, and fewer errors thanks to Jenkins.
Background: As a leading international bank, we wanted to gain faster development, better security, a less error-prone process, and have automated testing and document generation to maximize the time developers spent writing our packages.
Goals: Our main goal was to make producing and updating conda packages faster and easier so that we can better track the dependencies between packages and platforms.
Solution & Results: To gain faster development times, we used declarative Jenkinsfiles in our repositories so that Jenkins would automatically trigger builds after commits. Builds were triggered in Bitbucket so that developers needed only to write their code, and the package would build and upload if successful. After automated testing, code quality scanning, and conda upload stages, we wrote test stages to install the newly uploaded package from Artifactory in a separately provisioned container and then run tests on it. This is much easier than manually testing the package.
We use trunk-based development when conditions in our Jenkins stages let us have one Jenkinsfile but execute different stages based on which branch we are currently in. This allows us to control when something is ready to be released.
Jenkins lets us use technical accounts with credentials to perform actions such as upload packages to Artifactory or sonar code quality scanning. This means different users did not need releaser privileges to Artifactory, SonarQube, etc. This helped us reduce errors, limit the time wasted setting up permissions, and improve our security and audit.
We reduced build times, as Jenkins gave us the ability to provision agent nodes on Openshift for each build. This will potentially reduce environment-related errors as each build agent uses a new environment.
As for key capabilities relied on, the Sonarqube scanner plugin was a great help, as seeing scan results in Jenkins is very useful. The Artifactory plugin lets us upload/download artifacts as part of the pipeline without setting up credentials for each team member. Openshift plugin let us build and test in our cloud environment.
We are very pleased with the results, which include:
- Build and upload cycles reduced
- Documentation automatically updated
- Developer frustration reduced!