SQL Archives ~ fitzmoskal https://fitzmoskal.me/category/skills/tech-skills/sql/ You are _here_ Thu, 29 Feb 2024 18:13:53 +0000 en-GB hourly 1 https://wordpress.org/?v=6.9.4 https://fitzmoskal.me/wp-content/uploads/2024/02/cropped-result_880649-32x32.jpeg SQL Archives ~ fitzmoskal https://fitzmoskal.me/category/skills/tech-skills/sql/ 32 32 11678478 Egyptian Genomics Institute – Software Delivery https://fitzmoskal.me/egyptian-genomics-institute-software-delivery/ Mon, 26 Feb 2024 20:04:01 +0000 https://fitzmoskal.me/?p=1487 Software Delivery I managed the team that took part in a deployment of genomics analysis software to a government programme

The post Egyptian Genomics Institute – Software Delivery appeared first on fitzmoskal.

]]>

Software Delivery

I managed the team that took part in a deployment of genomics analysis software to a government programme based in Cairo Egypt worth £2m in total, I also acted as a hands-on technical lead.

I managed the resourcing of a consultant from China to attend for 3 weeks and had scheduled to be there on the first week. The deployment was time-sensitive and had to be completed on-premise in this window as we wouldn’t have remote access.

The consultant from China was delayed, so I had to use my technical skills to quickly get the install going having not performed one before. I did alone on the first day – a Sunday – and then brought in my other consultant and a DevOps resource on conference call for the following two days, until the Chinese consultant arrived. All this took place whilst the client was monitoring over my shoulder – so I had to present confidence in what I was doing at the same time. The software was successfully deployed inside of the 3 weeks. However there were some performance issues identifed.

Health Check

After concerns were raised by the client regarding performance – with accusations flying, I took the decision to prepare a mirror environment in our offices and we requested soem of their sample files to process and confirm that the software was capable of processing them – which it was. So the issues pointed to infrastructure.

I agreed to attend site with one of the product managers to perform a health check against the software and hardware. I had some input in what to look for from the DevOps team, but largely had to script the health check myself against Kubernetes, NFS and PostgreSQL. Onsite, I ran multiple tests and perfected the script which used a combination of tools including kubectl, docker, fio, iostat, mtr and netstat. With this I was able to narrow down the issue to packet loss between the application and NFS, which was hosted on a different network split by a switch or router – which was not as specified in the Solution Design we had outlined.

This helped to not only provide the customer with confidence in the solution and our commitment to support, but did prevent them from walking away from the contract as they had previously disputed ownership of the issue.

The post Egyptian Genomics Institute – Software Delivery appeared first on fitzmoskal.

]]>
1487
UK Telecoms Company – Digital Transformation https://fitzmoskal.me/uk-telecoms-company-digital-transformation/ Mon, 26 Feb 2024 15:45:35 +0000 https://fitzmoskal.me/?p=1462 As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>

As part of a larger digital transformation I was responsible for the architecture of their Discovery, modelling application architecture and data integration pipelines to the CMDB.

The initial deployment was minimal and did not cover the whole estate, which was airgapped. I was challenged to come up with an architecture which would be suitable enough capture data across the airgapped environments. The proposed solution involved manual DB snapshots which would be transferred by ftp, however this was later dropped in favour of deployment of a mutual shared platform, which was desperately needed.

This was a part-time multi-year engagement, during this time I developed and delivered various customatisations to their data collection and pipeline, such as:

  • SSL Certificate Discovery
  • Netcool edge device intergration
  • Service modelling
  • Reporting and analytics
  • Written maintenance manual

The post UK Telecoms Company – Digital Transformation appeared first on fitzmoskal.

]]>
1462
US Energy Company – CMDB Augmentation and Deep Database Architecture https://fitzmoskal.me/us-energy-company-cmdb-augmentation-and-deep-database-architecture/ Mon, 26 Feb 2024 15:10:40 +0000 https://fitzmoskal.me/?p=1456 I worked 3 short projects with a US-based energy company over a period of three years to develop customisations to

The post US Energy Company – CMDB Augmentation and Deep Database Architecture appeared first on fitzmoskal.

]]>

I worked 3 short projects with a US-based energy company over a period of three years to develop customisations to the data pipeline to their CMDB. This was to enable a front-end form for application ownership selection. For this they needed specifically to be able to extract metadata from their IBM DB2, MSSQL, Oracle, MongoDB and MariaDB databases for promotion to the front-end display.

The solution involved custom TPL code applied to BMC Discovery, which used a combination of SQL connections of JDBC and API connectivity with MongoDB. Both Discovery and the CMDB sync connections needed to be modified to make it work.

MongoDB presented the most challenging connection as Discovery was not built for a direct integration, so the solution called for running the API retrieval queries directly on the server as it scanned, with a local encrypted text file which contained the API credentials. The output was then retrieved the normal way.

The solution allowed for the client to be able to correctly identify and assign ownership for their database dependent applications.

The post US Energy Company – CMDB Augmentation and Deep Database Architecture appeared first on fitzmoskal.

]]>
1456
UK Non-profit – Datacenter Migration https://fitzmoskal.me/uk-charity-datacenter-migration/ Wed, 21 Feb 2024 15:27:47 +0000 https://fitzmoskal.me/?p=1395 A UK non-profit organisation planned datacenter move within the next 12 months. The required an understanding of their environment, the

The post UK Non-profit – Datacenter Migration appeared first on fitzmoskal.

]]>

A UK non-profit organisation planned datacenter move within the next 12 months. The required an understanding of their environment, the servers, applications and storage. Their main focus was on Production, DMZ and Disaster Recovery.

A successful proof of concept had been carried out prior and they were ready for deployment of the ITAM scanning solution.

A workshop was conducted to assess their capabilities and readiness as well as determine how the solution would be deployed. This results in a Solution Design and commencement of the deployment of scanning.

The client also required modelling of their application architecture in preparation, so they could understand the dependencies for the move. As well as delivering on these requirements within a six week timeframe, I was able to provide additional value by identifying approximately 100 Windows servers that were out of compliance.

The post UK Non-profit – Datacenter Migration appeared first on fitzmoskal.

]]>
1395
Online Betting Platforms – CMDB Data https://fitzmoskal.me/online-betting-platform-cmdb-data/ Tue, 20 Feb 2024 18:56:33 +0000 https://fitzmoskal.me/?p=1372 I was brought in to architect and deliver an data pipeline for updating their CMDBs from multiple data sources. The

The post Online Betting Platforms – CMDB Data appeared first on fitzmoskal.

]]>

I was brought in to architect and deliver an data pipeline for updating their CMDBs from multiple data sources. The CMDB itself was hosted on Salesforce and the data sources were Discovery, Active Directory, Spreadsheets and SCCM.

A particular challenge was being restricted to operating remotely using TeamViewer – which does not perform well over data instensive remote connections – and requires an active user on the end to keep the connection going. This did cause development work to go much slower than planned, but thankfully did not cause overall delay to project.

The integrations were developed, tested and deployed successfully to the satisfaction of the client.

The post Online Betting Platforms – CMDB Data appeared first on fitzmoskal.

]]>
1372
Major Fashion Retailer – Digital Transformation https://fitzmoskal.me/major-fashion-retailer-digital-transformation/ Tue, 20 Feb 2024 18:26:35 +0000 https://fitzmoskal.me/?p=1369 Over the years I built a relationship with this global fashion brand in facilitating their ongoing digital transformation. The first

The post Major Fashion Retailer – Digital Transformation appeared first on fitzmoskal.

]]>

Over the years I built a relationship with this global fashion brand in facilitating their ongoing digital transformation. The first engagement was in architecting and implementing a number of critical integrations with their CMDB from various data sources, allowing them to gain full central visiblity of their enterprise infrastructure. Later, I was again called upon to re-architect and extend the earlier integrations in preparation for a move of CMDB cloud platforms.

As a Principle Solutions Architect I also oversaw the technical delivery of their managed services – consulting on roadmaps and strategy with the client. I built a trusting working relationship and also proposed and worked on implementing a new feature of exception reporting, which helped them to identify, understand and narrow down the discrepencies between source data and CMDB data. The result was clear visibility and improved data quality of their CMDB.

The post Major Fashion Retailer – Digital Transformation appeared first on fitzmoskal.

]]>
1369
Investment Bank – ITAM & DB Migration https://fitzmoskal.me/investment-bank-itam/ Tue, 20 Feb 2024 15:57:32 +0000 https://fitzmoskal.me/?p=1321 I maintained a long-term relationship with a European Investment Bank based in Canary Wharf. As part of a team of

The post Investment Bank – ITAM & DB Migration appeared first on fitzmoskal.

]]>

I maintained a long-term relationship with a European Investment Bank based in Canary Wharf. As part of a team of consultants we had repeat work to maintain and upgrade their ITAM solution. I personally delivered on a 3-week critical project to facilitate a database migration. This involved liaising with lead DBAs to extract metadata from MS SQL, PostgreSQL and Sybase databases for the solution architecture.

This was a sensitive project given the job threat to the existing DBA SMEs. I faced open hostility from one SME in particular who’s tactic was to delay providing information and not respond to emails.

I resolved this with face to face contact, without letting any request go unanswered. I also used low-level escalation to my project sponsor. This did result in a tactical complaint from the SME to the project board, however my sponsor was satisfied with my approach, hitting their 3-week deadline and no fault could be found. I resolved not to make matters personal, which is how I managed to stay focused on delivery.

The post Investment Bank – ITAM & DB Migration appeared first on fitzmoskal.

]]>
1321
Middle Eastern Oil Company – CMDB ITAM Deployment https://fitzmoskal.me/me-oil-company-cmdb-itam-deployment/ Tue, 20 Feb 2024 15:04:19 +0000 https://fitzmoskal.me/?p=1307 I was assigned as a consultant to a large Middle Eastern Oil Company for six weeks onsite. BMC Discovery was

The post Middle Eastern Oil Company – CMDB ITAM Deployment appeared first on fitzmoskal.

]]>

I was assigned as a consultant to a large Middle Eastern Oil Company for six weeks onsite. BMC Discovery was their chosen IT Asset Management solution and their objectives were to deploy, scan their networks and model various services comprised of Oracle RAC Clusters, IBM LPARs and HMCs, Databases and add 2-way CMDB synchronisataion with custom asset locations.

The deployment and solution was delivered within the six week timeframe to satisfaction of the client. In addition they were supplied with various reports as value add. In one case they were able to identify a number of database servers which had been marked as decommissioned but not physically turned off. This improved their visibility and avoided ongoing costs and risk of exploited vulnerabilities in unmonitored infrastructure.

The post Middle Eastern Oil Company – CMDB ITAM Deployment appeared first on fitzmoskal.

]]>
1307
Spooning with Pentaho Data Integrator https://fitzmoskal.me/spooning-with-pentaho-data-integrator/ Fri, 14 Mar 2014 12:47:26 +0000 http://wafitz.net/?p=861 Pentaho Data Integrator is a neat open source, cross-platform tool for making light work of ETL. I have it set up

The post Spooning with Pentaho Data Integrator appeared first on fitzmoskal.

]]>
Pentaho Data Integrator is a neat open source, cross-platform tool for making light work of ETL. I have it set up on my Ubuntu laptop and can run it directly from the command line, or as part of a chron job.

However I found a couple of annoyances when running it from CLI, one was in having to keep a terminal window open, the other was having to run it from it’s install directory – particularly when it comes to relative path names for kettle jobs.

So I created an alias that runs PDI in an interactive shell allowing you to run it from one a word command, and it occurred to me that this might be useful to share. Here you go:

alias spoon='sh -c '\''cd /opt/pentaho/data-integration; ./spoon.sh&'\'''

 

The post Spooning with Pentaho Data Integrator appeared first on fitzmoskal.

]]>
861