DevOps Archives ~ fitzmoskal https://fitzmoskal.me/tag/devops/ You are _here_ Thu, 29 Feb 2024 18:13:53 +0000 en-GB hourly 1 https://wordpress.org/?v=6.9.4 https://fitzmoskal.me/wp-content/uploads/2024/02/cropped-result_880649-32x32.jpeg DevOps Archives ~ fitzmoskal https://fitzmoskal.me/tag/devops/ 32 32 11678478 Egyptian Genomics Institute – Software Delivery https://fitzmoskal.me/egyptian-genomics-institute-software-delivery/ Mon, 26 Feb 2024 20:04:01 +0000 https://fitzmoskal.me/?p=1487 Software Delivery I managed the team that took part in a deployment of genomics analysis software to a government programme

The post Egyptian Genomics Institute – Software Delivery appeared first on fitzmoskal.

]]>

Software Delivery

I managed the team that took part in a deployment of genomics analysis software to a government programme based in Cairo Egypt worth £2m in total, I also acted as a hands-on technical lead.

I managed the resourcing of a consultant from China to attend for 3 weeks and had scheduled to be there on the first week. The deployment was time-sensitive and had to be completed on-premise in this window as we wouldn’t have remote access.

The consultant from China was delayed, so I had to use my technical skills to quickly get the install going having not performed one before. I did alone on the first day – a Sunday – and then brought in my other consultant and a DevOps resource on conference call for the following two days, until the Chinese consultant arrived. All this took place whilst the client was monitoring over my shoulder – so I had to present confidence in what I was doing at the same time. The software was successfully deployed inside of the 3 weeks. However there were some performance issues identifed.

Health Check

After concerns were raised by the client regarding performance – with accusations flying, I took the decision to prepare a mirror environment in our offices and we requested soem of their sample files to process and confirm that the software was capable of processing them – which it was. So the issues pointed to infrastructure.

I agreed to attend site with one of the product managers to perform a health check against the software and hardware. I had some input in what to look for from the DevOps team, but largely had to script the health check myself against Kubernetes, NFS and PostgreSQL. Onsite, I ran multiple tests and perfected the script which used a combination of tools including kubectl, docker, fio, iostat, mtr and netstat. With this I was able to narrow down the issue to packet loss between the application and NFS, which was hosted on a different network split by a switch or router – which was not as specified in the Solution Design we had outlined.

This helped to not only provide the customer with confidence in the solution and our commitment to support, but did prevent them from walking away from the contract as they had previously disputed ownership of the issue.

The post Egyptian Genomics Institute – Software Delivery appeared first on fitzmoskal.

]]>
1487
NHS Genomics Partner – Integrations & Software Development https://fitzmoskal.me/nhs-genomics-partner-integrations-software-development/ Mon, 26 Feb 2024 19:31:19 +0000 https://fitzmoskal.me/?p=1484 As Head of Professional Services I took over responsibility for a number of software integrations written by a bioinformatics consultant

The post NHS Genomics Partner – Integrations & Software Development appeared first on fitzmoskal.

]]>

As Head of Professional Services I took over responsibility for a number of software integrations written by a bioinformatics consultant who had previously left the company. One of these integrations was supplied to a parnter organisation and had been reported to have an undetected design flaw which raised a ‘highest severity’ ticket. No-one in my team was familiar with the code, so I had to reach out and coordinate with DevOps and Engineering for assistance.

Through bargaining and persistance I secured some time of one of the lead bioinformaticians to take a look at the code, review the clients requirements and make the changes. This was the easiest part – I recognised the issue with the code myself, but as I wouldn’t understand the genomics interpretation part, I did not want to make edits myself. The second part of the issue was with clinical decisions that had already been made with the flawed code. There was little that I could do here, but it was down to me to make sure that the right resources were aware and assigned to figure out which decisions had been impacted to report back to the client.

The code fix was done within SLAs, finding the impacted clinical decisions took longer. My final action was to ensure that the application was correctly documented for future support and ensure that the client sign off on the new implementation of the code (to ensure that there is record of agreement that it now worked as expected – which was missing before).

I was also engaged with this partner through organising a resource to work on custom reporting, using a combination of python and flask. I acted as liaison and project manager with the customer ensuring that the code was developed to specification and that milestones for delivery, test and sign off were met.

The post NHS Genomics Partner – Integrations & Software Development appeared first on fitzmoskal.

]]>
1484
US Retailer – DevOps https://fitzmoskal.me/us-retailer-devops/ Mon, 26 Feb 2024 17:59:58 +0000 https://fitzmoskal.me/?p=1481 I was asked to provide a health check with insights into the clients Discovery operations. I created a custom application

The post US Retailer – DevOps appeared first on fitzmoskal.

]]>

I was asked to provide a health check with insights into the clients Discovery operations. I created a custom application called Pithos – a python-based application which collected data from CMDB and Discovery. This data was then imported into Power BI and we were able to present the following back to the client:

  • High disk utilisation
  • Presence of core dumps
  • Segregated architecture, but no network segregation
  • Sensitive data such as plaintext usernames and passwords stored in unstructured data fields
  • Scan schedules impacting performance

This was followed up with a roadmap of improvements for the client with the option of being delivered by my company.

The post US Retailer – DevOps appeared first on fitzmoskal.

]]>
1481
UK Pharmaceutical – DevOps https://fitzmoskal.me/uk-pharmaceutical-aiops/ Mon, 26 Feb 2024 17:47:33 +0000 https://fitzmoskal.me/?p=1478 I worked with presales to deliver a $55k project with insights into the clients Discovery operations created with Pithos –

The post UK Pharmaceutical – DevOps appeared first on fitzmoskal.

]]>

I worked with presales to deliver a $55k project with insights into the clients Discovery operations created with Pithos – a custom built python-based application which collected data from CMDB and Discovery. This data was then imported into Power BI and we were able to present the following back to the client:

  • Sensitive data filters not capturing plaintext username/password
  • Under-specced Outpost sizes causing performance issues
  • Active user accounts from departed users not removed
  • Various data quality issues
  • Configuration discrepencies between prod and dev environments

This was followed up with a roadmap of improvements for the client which included development of a runbook, and 10 consultancy days of remediation.

The post UK Pharmaceutical – DevOps appeared first on fitzmoskal.

]]>
1478
UK Telecoms Company – Digital Transformation https://fitzmoskal.me/uk-telecoms-company-digital-transformation/ Mon, 26 Feb 2024 15:45:35 +0000 https://fitzmoskal.me/?p=1462 As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>

As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and data integration pipelines to the CMDB.

The initial deployment was minimal and did not cover the whole estate, which was airgapped. I was challenged to come up with an architecture which would be suitable enough capture data across the airgapped environments. The proposed solution involved manual DB snapshots which would be transferred by ftp, however this was later dropped in favour of deployment of a mutual shared platform, which was desperately needed.

This was a part-time multi-year engagement, during this time I developed and delivered various customatisations to their data collection and pipeline, such as:

  • SSL Certificate Discovery
  • Netcool edge device intergration
  • Service modelling
  • Reporting and analytics
  • Written maintenance manual

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>
1462
UK Retailer – Managed Service Architecture https://fitzmoskal.me/uk-retailer-managed-service-architecture/ Mon, 26 Feb 2024 15:29:31 +0000 https://fitzmoskal.me/?p=1459 I was responsible for the improving the managed service architecture of a multi-year service contract already in-flight. There had been

The post UK Retailer – Managed Service Architecture appeared first on fitzmoskal.

]]>

I was responsible for the improving the managed service architecture of a multi-year service contract already in-flight. There had been significant issues raised over the quality of the service up until my engagemement.

The main complaints were:

  • Support resource not seen to be solving any reported issues
  • Missing roadmap meetings
  • Not feeling the value

An action plan and strategy was put in place, for me to conduct quarterly service reviews where I would present the latest features of the products and recommend strategy for upgrade and utilisation of these features, including any enhancements we could provide to the service.

The support resource issue was identified as being a follow on outcome of the customer not willing to log tickets and instead engaging the named resource directly. In this way the resource was being treated as a contractor and being asked to work on various unspecified tasks and issues. In discussing the issue with the client, I suggested they could either continue with no SLAs and agree to the support resource under 1-day per week effort, or that in order for us to track performance and ensure requests were deailt with, they would need to engage with the service and log tickets.

However, as many of the requests would have been out of the scope of the service and meant engagement with professional services, they did not want to implement this change. They prefered to carry on under the existing arrangement, without further complaint.

During this time I was also able to implement health checks for them and identify a number of security vulnerabilites which were esclated to their CISO. One was the identification of plaintext username/password strings in running processes, and the other was production level software running under a non-service account credential.

The post UK Retailer – Managed Service Architecture appeared first on fitzmoskal.

]]>
1459
US Energy Company – CMDB Augmentation and Deep Database Architecture https://fitzmoskal.me/us-energy-company-cmdb-augmentation-and-deep-database-architecture/ Mon, 26 Feb 2024 15:10:40 +0000 https://fitzmoskal.me/?p=1456 I worked 3 short projects with a US-based energy company over a period of three years to develop customisations to

The post US Energy Company – CMDB Augmentation and Deep Database Architecture appeared first on fitzmoskal.

]]>

I worked 3 short projects with a US-based energy company over a period of three years to develop customisations to the data pipeline to their CMDB. This was to enable a front-end form for application ownership selection. For this they needed specifically to be able to extract metadata from their IBM DB2, MSSQL, Oracle, MongoDB and MariaDB databases for promotion to the front-end display.

The solution involved custom TPL code applied to BMC Discovery, which used a combination of SQL connections of JDBC and API connectivity with MongoDB. Both Discovery and the CMDB sync connections needed to be modified to make it work.

MongoDB presented the most challenging connection as Discovery was not built for a direct integration, so the solution called for running the API retrieval queries directly on the server as it scanned, with a local encrypted text file which contained the API credentials. The output was then retrieved the normal way.

The solution allowed for the client to be able to correctly identify and assign ownership for their database dependent applications.

The post US Energy Company – CMDB Augmentation and Deep Database Architecture appeared first on fitzmoskal.

]]>
1456
US Vehicle Manufacturer – Managed Service https://fitzmoskal.me/us-vehicle-manufacturer-managed-service/ Sat, 24 Feb 2024 02:54:49 +0000 https://fitzmoskal.me/?p=1452 I was contracted to help win a bid for providing a managed service worth $200k to a large US-based vehicle

The post US Vehicle Manufacturer – Managed Service appeared first on fitzmoskal.

]]>

I was contracted to help win a bid for providing a managed service worth $200k to a large US-based vehicle manufacturer. They were intially discussing a multi-year deal for plant inventory discovery, however it was quickly realised by the client that it would make sense to turn this into a managed service.

The service included upgrade of current Discovery solution with right-sizing the architecture, installation of Remedy with CMDB and support services. The scope of Discovery was to scan the plant networks – the challenge being that these were on profinet and were not only sensitive equipment but scanning was only permitted under very strict windows.

We managed to complete the implementation of the solution upgrade and installation, with the implementation of the full managed service coming later due to delays on the client end. As part of value add of the service, the customer was also provided with a custom solution I developed in python that worked with the Discovery API to be able to load IP ranges with updated location information and automatically update the scan data and schedules.

The post US Vehicle Manufacturer – Managed Service appeared first on fitzmoskal.

]]>
1452