Python Archives ~ fitzmoskal https://fitzmoskal.me/category/technologies/expert/python-technologies/ You are _here_ Tue, 05 Mar 2024 20:59:04 +0000 en-GB hourly 1 https://wordpress.org/?v=6.9.4 https://fitzmoskal.me/wp-content/uploads/2024/02/cropped-result_880649-32x32.jpeg Python Archives ~ fitzmoskal https://fitzmoskal.me/category/technologies/expert/python-technologies/ 32 32 11678478 RAG with Offline LLM https://fitzmoskal.me/rag-with-offline-llm/ Tue, 05 Mar 2024 20:59:03 +0000 https://fitzmoskal.me/?p=1755 During my time as Head of Professional Services for a private Genomics based company I spearheaded an initiative to utilise

The post RAG with Offline LLM appeared first on fitzmoskal.

]]>

During my time as Head of Professional Services for a private Genomics based company I spearheaded an initiative to utilise an open source offline LLM for analysing our Jira, Confluence and Salesforce docs for quick retrieval of process and technical information.

The purpose was twofold,

a) To create a common ‘search’ tool in order to locate a source of truth as to what was configured and where, and

b) To be able to generate process documents very quickly in whole or in part through the use of generative AI.

As these were sensitive company documents, use of a service like ChatGPT was restricted, which is why we opted to trial PrivateGPT with Llama-2-7b amongst other models.

We found RAG to be less than perfect, but it did effectively allow us to search out the right documents. Text generation was better in that we managed to produce good content as an initial template for a process guides, as well as feeding a full process document to generate a RASCI chart. We ran the pilot service on AWS EC2 infrastruture.

The post RAG with Offline LLM appeared first on fitzmoskal.

]]>
1755
VCF File Validation https://fitzmoskal.me/vcf-file-validation/ Mon, 26 Feb 2024 21:08:20 +0000 https://fitzmoskal.me/?p=1497 This script was bourne of necessity to validate VCF files for injestion into Congenica. The platform does it’s own validation

The post VCF File Validation appeared first on fitzmoskal.

]]>
This script was bourne of necessity to validate VCF files for injestion into Congenica. The platform does it’s own validation but the output in case of error is not human-friendly. This directs the user to which line in the VCF file might be at fault.

https://github.com/codefitz/vcf_validation

The post VCF File Validation appeared first on fitzmoskal.

]]>
1497
Kubespray Dependencies https://fitzmoskal.me/kubespray-dependencies/ Mon, 26 Feb 2024 20:31:46 +0000 https://fitzmoskal.me/?p=1495 Anyone who has worked witk Kubespray knows what kind of hell it is to figure out the dependencies. Part of

The post Kubespray Dependencies appeared first on fitzmoskal.

]]>
Anyone who has worked witk Kubespray knows what kind of hell it is to figure out the dependencies. Part of the reason for this is that each version is kept on a seperate branch, and each branch will have it’s own buried readme – which notes the particular version of Python and Ansible is supported. Easy when you know how, but a pain if you’re dealing with complex installation processes and installing dependencies for the first time.

This script I wrote essentially takes a python version or an ansible version and will loop through checking out each branch then printout which versions of Kubespray are supported.

https://github.com/codefitz/kubespray-dependencies

The post Kubespray Dependencies appeared first on fitzmoskal.

]]>
1495
NHS Genomics Partner – Integrations & Software Development https://fitzmoskal.me/nhs-genomics-partner-integrations-software-development/ Mon, 26 Feb 2024 19:31:19 +0000 https://fitzmoskal.me/?p=1484 As Head of Professional Services I took over responsibility for a number of software integrations written by a bioinformatics consultant

The post NHS Genomics Partner – Integrations & Software Development appeared first on fitzmoskal.

]]>

As Head of Professional Services I took over responsibility for a number of software integrations written by a bioinformatics consultant who had previously left the company. One of these integrations was supplied to a parnter organisation and had been reported to have an undetected design flaw which raised a ‘highest severity’ ticket. No-one in my team was familiar with the code, so I had to reach out and coordinate with DevOps and Engineering for assistance.

Through bargaining and persistance I secured some time of one of the lead bioinformaticians to take a look at the code, review the clients requirements and make the changes. This was the easiest part – I recognised the issue with the code myself, but as I wouldn’t understand the genomics interpretation part, I did not want to make edits myself. The second part of the issue was with clinical decisions that had already been made with the flawed code. There was little that I could do here, but it was down to me to make sure that the right resources were aware and assigned to figure out which decisions had been impacted to report back to the client.

The code fix was done within SLAs, finding the impacted clinical decisions took longer. My final action was to ensure that the application was correctly documented for future support and ensure that the client sign off on the new implementation of the code (to ensure that there is record of agreement that it now worked as expected – which was missing before).

I was also engaged with this partner through organising a resource to work on custom reporting, using a combination of python and flask. I acted as liaison and project manager with the customer ensuring that the code was developed to specification and that milestones for delivery, test and sign off were met.

The post NHS Genomics Partner – Integrations & Software Development appeared first on fitzmoskal.

]]>
1484
US Retailer – DevOps https://fitzmoskal.me/us-retailer-devops/ Mon, 26 Feb 2024 17:59:58 +0000 https://fitzmoskal.me/?p=1481 I was asked to provide a health check with insights into the clients Discovery operations. I created a custom application

The post US Retailer – DevOps appeared first on fitzmoskal.

]]>

I was asked to provide a health check with insights into the clients Discovery operations. I created a custom application called Pithos – a python-based application which collected data from CMDB and Discovery. This data was then imported into Power BI and we were able to present the following back to the client:

  • High disk utilisation
  • Presence of core dumps
  • Segregated architecture, but no network segregation
  • Sensitive data such as plaintext usernames and passwords stored in unstructured data fields
  • Scan schedules impacting performance

This was followed up with a roadmap of improvements for the client with the option of being delivered by my company.

The post US Retailer – DevOps appeared first on fitzmoskal.

]]>
1481
UK Pharmaceutical – DevOps https://fitzmoskal.me/uk-pharmaceutical-aiops/ Mon, 26 Feb 2024 17:47:33 +0000 https://fitzmoskal.me/?p=1478 I worked with presales to deliver a $55k project with insights into the clients Discovery operations created with Pithos –

The post UK Pharmaceutical – DevOps appeared first on fitzmoskal.

]]>

I worked with presales to deliver a $55k project with insights into the clients Discovery operations created with Pithos – a custom built python-based application which collected data from CMDB and Discovery. This data was then imported into Power BI and we were able to present the following back to the client:

  • Sensitive data filters not capturing plaintext username/password
  • Under-specced Outpost sizes causing performance issues
  • Active user accounts from departed users not removed
  • Various data quality issues
  • Configuration discrepencies between prod and dev environments

This was followed up with a roadmap of improvements for the client which included development of a runbook, and 10 consultancy days of remediation.

The post UK Pharmaceutical – DevOps appeared first on fitzmoskal.

]]>
1478
Healthcare Company – AIOps https://fitzmoskal.me/healthcare-company-aiops/ Mon, 26 Feb 2024 17:29:05 +0000 https://fitzmoskal.me/?p=1475 I worked in presales to present insights into their ITSM operations gathered from Talos back to the client. My involvement

The post Healthcare Company – AIOps appeared first on fitzmoskal.

]]>

I worked in presales to present insights into their ITSM operations gathered from Talos back to the client. My involvement was in data cleansing and preparation for injesting into Talos for supervised and unsupervised learning. Once Talos had created the reports, I would then import these to PowerBI to develop the insights. With AI we were able to identify:

  • Issue candidates for automation e.g. “password reset”
  • Lack of categorisation and service catalogue, poor summaries written in tickets
  • 3rd party structured automation information posted into notes fields
  • Categorical data located in unstructured notes fields
  • Under-utilisation of knowledge
  • Change requests logged as tickets leading to high level of tickets with low MTTR
  • High level of critical abnormalities requiring problem RCA
  • General lack of granularity in categories, impacting management reporting

This was followed up with an offer to the client to deliver the recommendations such as service desk training, restructuring categories, data cleansing, and knowledgebase improvements

The post Healthcare Company – AIOps appeared first on fitzmoskal.

]]>
1475
UK Telecoms Company – Digital Transformation https://fitzmoskal.me/uk-telecoms-company-digital-transformation/ Mon, 26 Feb 2024 15:45:35 +0000 https://fitzmoskal.me/?p=1462 As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>

As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and data integration pipelines to the CMDB.

The initial deployment was minimal and did not cover the whole estate, which was airgapped. I was challenged to come up with an architecture which would be suitable enough capture data across the airgapped environments. The proposed solution involved manual DB snapshots which would be transferred by ftp, however this was later dropped in favour of deployment of a mutual shared platform, which was desperately needed.

This was a part-time multi-year engagement, during this time I developed and delivered various customatisations to their data collection and pipeline, such as:

  • SSL Certificate Discovery
  • Netcool edge device intergration
  • Service modelling
  • Reporting and analytics
  • Written maintenance manual

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>
1462
German IT Consulting Firm – Digital Transformation https://fitzmoskal.me/german-it-consulting-firm-digital-transformation/ Thu, 22 Feb 2024 16:59:03 +0000 https://fitzmoskal.me/?p=1437 I was in charge of the architecture for the deployment £200k worth of solutions and a managed service for a

The post German IT Consulting Firm – Digital Transformation appeared first on fitzmoskal.

]]>
This image has an empty alt attribute; its file name is hamburg-3071437_1280-1024x682.jpg

I was in charge of the architecture for the deployment £200k worth of solutions and a managed service for a large German consulting firm that wanted to implement a new Asset & Configuration Management function using BMC’s toolset.

Once the deployment was completed and the customer onboarded, I acted as an 3rd line escalation point issues logged through the managed service. In particular I got involved with issues with Data Quality and modelling their network environments. As part of the service they were also gauranteed regular health checks – which we provided in the form of a unique set of analytics written in Python which were exported to Power BI for presenting insights back to the customer, such as:

  • Root user logins
  • Vulernable open ports
  • Recommendations for search and query optimisation
  • Compliance issues
  • Data quality issues

This unique service was provided only by the company I was working for at the time where I developed it.

The post German IT Consulting Firm – Digital Transformation appeared first on fitzmoskal.

]]>
1437
US Coffee Company – SSL Certificate Discovery https://fitzmoskal.me/us-coffee-company-ssl-certificate-discovery/ Thu, 22 Feb 2024 15:18:23 +0000 https://fitzmoskal.me/?p=1429 I was approached by the account management team for a US coffee brand to implement SSL Certificate Discovery for their

The post US Coffee Company – SSL Certificate Discovery appeared first on fitzmoskal.

]]>

I was approached by the account management team for a US coffee brand to implement SSL Certificate Discovery for their network, as they were having issues with expiring certificates causing application failures.

During deployment one challenge was the need to expand the tool to query the Windows stores and resolve an issue where Azure agents would automatically issue and manage short-term 24 hour expiries – which significantly skewed the analytics. These certificates were identified and excluded from reporting.

The solution was deployed quickly and there was time to develop additional analytics and dashboards to allow the account team to quickly export a list of all certificate expiry dates.

The post US Coffee Company – SSL Certificate Discovery appeared first on fitzmoskal.

]]>
1429