Scripting Archives ~ fitzmoskal https://fitzmoskal.me/category/skills/tech-skills/scripting/ You are _here_ Tue, 05 Mar 2024 20:59:04 +0000 en-GB hourly 1 https://wordpress.org/?v=6.9.4 https://fitzmoskal.me/wp-content/uploads/2024/02/cropped-result_880649-32x32.jpeg Scripting Archives ~ fitzmoskal https://fitzmoskal.me/category/skills/tech-skills/scripting/ 32 32 11678478 RAG with Offline LLM https://fitzmoskal.me/rag-with-offline-llm/ Tue, 05 Mar 2024 20:59:03 +0000 https://fitzmoskal.me/?p=1755 During my time as Head of Professional Services for a private Genomics based company I spearheaded an initiative to utilise

The post RAG with Offline LLM appeared first on fitzmoskal.

]]>

During my time as Head of Professional Services for a private Genomics based company I spearheaded an initiative to utilise an open source offline LLM for analysing our Jira, Confluence and Salesforce docs for quick retrieval of process and technical information.

The purpose was twofold,

a) To create a common ‘search’ tool in order to locate a source of truth as to what was configured and where, and

b) To be able to generate process documents very quickly in whole or in part through the use of generative AI.

As these were sensitive company documents, use of a service like ChatGPT was restricted, which is why we opted to trial PrivateGPT with Llama-2-7b amongst other models.

We found RAG to be less than perfect, but it did effectively allow us to search out the right documents. Text generation was better in that we managed to produce good content as an initial template for a process guides, as well as feeding a full process document to generate a RASCI chart. We ran the pilot service on AWS EC2 infrastruture.

The post RAG with Offline LLM appeared first on fitzmoskal.

]]>
1755
VCF File Validation https://fitzmoskal.me/vcf-file-validation/ Mon, 26 Feb 2024 21:08:20 +0000 https://fitzmoskal.me/?p=1497 This script was bourne of necessity to validate VCF files for injestion into Congenica. The platform does it’s own validation

The post VCF File Validation appeared first on fitzmoskal.

]]>
This script was bourne of necessity to validate VCF files for injestion into Congenica. The platform does it’s own validation but the output in case of error is not human-friendly. This directs the user to which line in the VCF file might be at fault.

https://github.com/codefitz/vcf_validation

The post VCF File Validation appeared first on fitzmoskal.

]]>
1497
Kubespray Dependencies https://fitzmoskal.me/kubespray-dependencies/ Mon, 26 Feb 2024 20:31:46 +0000 https://fitzmoskal.me/?p=1495 Anyone who has worked witk Kubespray knows what kind of hell it is to figure out the dependencies. Part of

The post Kubespray Dependencies appeared first on fitzmoskal.

]]>
Anyone who has worked witk Kubespray knows what kind of hell it is to figure out the dependencies. Part of the reason for this is that each version is kept on a seperate branch, and each branch will have it’s own buried readme – which notes the particular version of Python and Ansible is supported. Easy when you know how, but a pain if you’re dealing with complex installation processes and installing dependencies for the first time.

This script I wrote essentially takes a python version or an ansible version and will loop through checking out each branch then printout which versions of Kubespray are supported.

https://github.com/codefitz/kubespray-dependencies

The post Kubespray Dependencies appeared first on fitzmoskal.

]]>
1495
German Car Systems Maker – Storage Integration https://fitzmoskal.me/german-car-systems-maker-storage-integration/ Mon, 26 Feb 2024 16:31:21 +0000 https://fitzmoskal.me/?p=1468 I worked with this German company to enhance their Discovery and CMDB with various storage metadata and data from unsupported

The post German Car Systems Maker – Storage Integration appeared first on fitzmoskal.

]]>

I worked with this German company to enhance their Discovery and CMDB with various storage metadata and data from unsupported storage systems. I wrote extensive TPL code which integrated with APIs for NetApp, Dell Isilon, IBM Spetrum, and Nasuni.

For Nasuni I had to develop the whole model using data extracted from the APIs to model the Management Console, Volumes and Shares.

The outcome of this project was to allow the client to accurately track storage usage and billing.

During this time I also assisted the client in troubleshooting Azure discovery and retrieving additional matadata from VMware.

The post German Car Systems Maker – Storage Integration appeared first on fitzmoskal.

]]>
1468
UK Telecoms Company – Digital Transformation https://fitzmoskal.me/uk-telecoms-company-digital-transformation/ Mon, 26 Feb 2024 15:45:35 +0000 https://fitzmoskal.me/?p=1462 As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>

As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and data integration pipelines to the CMDB.

The initial deployment was minimal and did not cover the whole estate, which was airgapped. I was challenged to come up with an architecture which would be suitable enough capture data across the airgapped environments. The proposed solution involved manual DB snapshots which would be transferred by ftp, however this was later dropped in favour of deployment of a mutual shared platform, which was desperately needed.

This was a part-time multi-year engagement, during this time I developed and delivered various customatisations to their data collection and pipeline, such as:

  • SSL Certificate Discovery
  • Netcool edge device intergration
  • Service modelling
  • Reporting and analytics
  • Written maintenance manual

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>
1462
US Energy Company – CMDB Augmentation and Deep Database Architecture https://fitzmoskal.me/us-energy-company-cmdb-augmentation-and-deep-database-architecture/ Mon, 26 Feb 2024 15:10:40 +0000 https://fitzmoskal.me/?p=1456 I worked 3 short projects with a US-based energy company over a period of three years to develop customisations to

The post US Energy Company – CMDB Augmentation and Deep Database Architecture appeared first on fitzmoskal.

]]>

I worked 3 short projects with a US-based energy company over a period of three years to develop customisations to the data pipeline to their CMDB. This was to enable a front-end form for application ownership selection. For this they needed specifically to be able to extract metadata from their IBM DB2, MSSQL, Oracle, MongoDB and MariaDB databases for promotion to the front-end display.

The solution involved custom TPL code applied to BMC Discovery, which used a combination of SQL connections of JDBC and API connectivity with MongoDB. Both Discovery and the CMDB sync connections needed to be modified to make it work.

MongoDB presented the most challenging connection as Discovery was not built for a direct integration, so the solution called for running the API retrieval queries directly on the server as it scanned, with a local encrypted text file which contained the API credentials. The output was then retrieved the normal way.

The solution allowed for the client to be able to correctly identify and assign ownership for their database dependent applications.

The post US Energy Company – CMDB Augmentation and Deep Database Architecture appeared first on fitzmoskal.

]]>
1456
US Vehicle Manufacturer – Managed Service https://fitzmoskal.me/us-vehicle-manufacturer-managed-service/ Sat, 24 Feb 2024 02:54:49 +0000 https://fitzmoskal.me/?p=1452 I was contracted to help win a bid for providing a managed service worth $200k to a large US-based vehicle

The post US Vehicle Manufacturer – Managed Service appeared first on fitzmoskal.

]]>

I was contracted to help win a bid for providing a managed service worth $200k to a large US-based vehicle manufacturer. They were intially discussing a multi-year deal for plant inventory discovery, however it was quickly realised by the client that it would make sense to turn this into a managed service.

The service included upgrade of current Discovery solution with right-sizing the architecture, installation of Remedy with CMDB and support services. The scope of Discovery was to scan the plant networks – the challenge being that these were on profinet and were not only sensitive equipment but scanning was only permitted under very strict windows.

We managed to complete the implementation of the solution upgrade and installation, with the implementation of the full managed service coming later due to delays on the client end. As part of value add of the service, the customer was also provided with a custom solution I developed in python that worked with the Discovery API to be able to load IP ranges with updated location information and automatically update the scan data and schedules.

The post US Vehicle Manufacturer – Managed Service appeared first on fitzmoskal.

]]>
1452
US Coffee Company – SSL Certificate Discovery https://fitzmoskal.me/us-coffee-company-ssl-certificate-discovery/ Thu, 22 Feb 2024 15:18:23 +0000 https://fitzmoskal.me/?p=1429 I was approached by the account management team for a US coffee brand to implement SSL Certificate Discovery for their

The post US Coffee Company – SSL Certificate Discovery appeared first on fitzmoskal.

]]>

I was approached by the account management team for a US coffee brand to implement SSL Certificate Discovery for their network, as they were having issues with expiring certificates causing application failures.

During deployment one challenge was the need to expand the tool to query the Windows stores and resolve an issue where Azure agents would automatically issue and manage short-term 24 hour expiries – which significantly skewed the analytics. These certificates were identified and excluded from reporting.

The solution was deployed quickly and there was time to develop additional analytics and dashboards to allow the account team to quickly export a list of all certificate expiry dates.

The post US Coffee Company – SSL Certificate Discovery appeared first on fitzmoskal.

]]>
1429
US Investment Bank – SLM & SAN Storage https://fitzmoskal.me/us-investment-bank-slm-san-storage/ Thu, 22 Feb 2024 13:21:12 +0000 https://fitzmoskal.me/?p=1414 I worked for a partner consultantancy with the Software and Configuration Management team to extend and customise BMC Discovery to

The post US Investment Bank – SLM & SAN Storage appeared first on fitzmoskal.

]]>

I worked for a partner consultantancy with the Software and Configuration Management team to extend and customise BMC Discovery to provide support for their licensing and auditing function. With the software license discovery and modelling that I workend on both indendently and sometimes as part of a team, the bank was able to make substantial savings in the 100’s of thousands of US Dollars on licensing to vendors such as Oracle, IBM and Microsoft.

I worked under and Agile framework to develop TPL code, performed monthly regression analysis on software install data and created and maintained all documentation for customisations.

I was also part of the team to code and develop an extension that modelled their various SAN storage solutions deployed. After the solution was implemented, the bank was able to identify 10,000 US Dollars in monthly savings by being able to turn off a decommission storage arrays that were no longer being used.

The post US Investment Bank – SLM & SAN Storage appeared first on fitzmoskal.

]]>
1414
US Investment Bank – Edge Device Discovery https://fitzmoskal.me/us-investment-bank-edge-device-discovery/ Wed, 21 Feb 2024 14:45:25 +0000 https://fitzmoskal.me/?p=1388 A leading US Investment Bank wanted to extend their off-the-shelf scanning solution to discover edge devices and follow hops to

The post US Investment Bank – Edge Device Discovery appeared first on fitzmoskal.

]]>

A leading US Investment Bank wanted to extend their off-the-shelf scanning solution to discover edge devices and follow hops to find the very edge of the network. This was not possible OOTB.

I developed a small solution comprising of shell scripting and TPL to discover and map the edge devices which allowed identification of unmonitored networks within the Bank’s infrastructure. The output was sent to CSV files which was the format requested for injestion into their network monitoring tool.

I also wrote a TPL file to discover Windows disk drives, which was another requirement from the client. I finished up by writing full documentation on the approach and code before handing over to the client.

The post US Investment Bank – Edge Device Discovery appeared first on fitzmoskal.

]]>
1388