Congenica Archives ~ fitzmoskal

VCF File Validation

Wes — Mon, 26 Feb 2024 21:08:20 +0000

This script was bourne of necessity to validate VCF files for injestion into Congenica. The platform does it’s own validation but the output in case of error is not human-friendly. This directs the user to which line in the VCF file might be at fault.

https://github.com/codefitz/vcf_validation

The post VCF File Validation appeared first on fitzmoskal.

Kubespray Dependencies

Wes — Mon, 26 Feb 2024 20:31:46 +0000

Anyone who has worked witk Kubespray knows what kind of hell it is to figure out the dependencies. Part of the reason for this is that each version is kept on a seperate branch, and each branch will have it’s own buried readme – which notes the particular version of Python and Ansible is supported. Easy when you know how, but a pain if you’re dealing with complex installation processes and installing dependencies for the first time.

This script I wrote essentially takes a python version or an ansible version and will loop through checking out each branch then printout which versions of Kubespray are supported.

https://github.com/codefitz/kubespray-dependencies

The post Kubespray Dependencies appeared first on fitzmoskal.

Egyptian Genomics Institute – Software Delivery

Wes — Mon, 26 Feb 2024 20:04:01 +0000

Software Delivery

I managed the team that took part in a deployment of genomics analysis software to a government programme based in Cairo Egypt worth £2m in total, I also acted as a hands-on technical lead.

I managed the resourcing of a consultant from China to attend for 3 weeks and had scheduled to be there on the first week. The deployment was time-sensitive and had to be completed on-premise in this window as we wouldn’t have remote access.

The consultant from China was delayed, so I had to use my technical skills to quickly get the install going having not performed one before. I did alone on the first day – a Sunday – and then brought in my other consultant and a DevOps resource on conference call for the following two days, until the Chinese consultant arrived. All this took place whilst the client was monitoring over my shoulder – so I had to present confidence in what I was doing at the same time. The software was successfully deployed inside of the 3 weeks. However there were some performance issues identifed.

Health Check

After concerns were raised by the client regarding performance – with accusations flying, I took the decision to prepare a mirror environment in our offices and we requested soem of their sample files to process and confirm that the software was capable of processing them – which it was. So the issues pointed to infrastructure.

I agreed to attend site with one of the product managers to perform a health check against the software and hardware. I had some input in what to look for from the DevOps team, but largely had to script the health check myself against Kubernetes, NFS and PostgreSQL. Onsite, I ran multiple tests and perfected the script which used a combination of tools including kubectl, docker, fio, iostat, mtr and netstat. With this I was able to narrow down the issue to packet loss between the application and NFS, which was hosted on a different network split by a switch or router – which was not as specified in the Solution Design we had outlined.

This helped to not only provide the customer with confidence in the solution and our commitment to support, but did prevent them from walking away from the contract as they had previously disputed ownership of the issue.

The post Egyptian Genomics Institute – Software Delivery appeared first on fitzmoskal.

NHS Genomics Partner – Integrations & Software Development

Wes — Mon, 26 Feb 2024 19:31:19 +0000

As Head of Professional Services I took over responsibility for a number of software integrations written by a bioinformatics consultant who had previously left the company. One of these integrations was supplied to a parnter organisation and had been reported to have an undetected design flaw which raised a ‘highest severity’ ticket. No-one in my team was familiar with the code, so I had to reach out and coordinate with DevOps and Engineering for assistance.

Through bargaining and persistance I secured some time of one of the lead bioinformaticians to take a look at the code, review the clients requirements and make the changes. This was the easiest part – I recognised the issue with the code myself, but as I wouldn’t understand the genomics interpretation part, I did not want to make edits myself. The second part of the issue was with clinical decisions that had already been made with the flawed code. There was little that I could do here, but it was down to me to make sure that the right resources were aware and assigned to figure out which decisions had been impacted to report back to the client.

The code fix was done within SLAs, finding the impacted clinical decisions took longer. My final action was to ensure that the application was correctly documented for future support and ensure that the client sign off on the new implementation of the code (to ensure that there is record of agreement that it now worked as expected – which was missing before).

I was also engaged with this partner through organising a resource to work on custom reporting, using a combination of python and flask. I acted as liaison and project manager with the customer ensuring that the code was developed to specification and that milestones for delivery, test and sign off were met.

The post NHS Genomics Partner – Integrations & Software Development appeared first on fitzmoskal.

Digital Health Checks

Wes — Tue, 20 Feb 2024 19:37:14 +0000

During my time as a consultant I have worked to develop various digital health checks to assist companies with taking a tempurature test of their IT systems and strategy. The view of health checks is sometimes that they are just a tool to drum up more business – which to an extent is true, but the health checks I developed have been critical in assisting companies to identify risks, avoid costly mistakes and pinpoint the source of complaints.

I have developed comprehensive health checks for BMC Discovery, Kubernetes, NFS and PostgreSQL. I have also developed exception reporting on data pipelines for data quality assessment. The health checks I have developed have saved customers in the following areas:

Identified security vulnerabilities around plaintext usernames and passwords storage
Identified issues around NFS performance
Identified opportunities for license reclaimation
Identified usage patterns for insecure practices (i.e. logging in as root user)
Identified various misconfigurations in non-alignment with best practices

The post Digital Health Checks appeared first on fitzmoskal.

Sample Sizing

Wes — Mon, 19 Feb 2024 20:40:21 +0000

A small script utilising scikit-learn for predicting the size of a build of Congenica on-premise given the number of genome samples to process.

https://github.com/codefitz/sample-sizing

The post Sample Sizing appeared first on fitzmoskal.