DISMAL ~ fitzmoskal

Toolkit built for use with BMC Discovery written in Python. Can do a number of powerful reports and modifications with both API and CLI (appliance) based commands.

DISMAL – Discovery-Mod-And-Lookup Toolkit
https://github.com/codefitz/DisMAL
0 forks.
0 stars.
0 open issues.

Recent commits:

v0.3.6 (#279)* Refactor to use tideway API for active scan queriesReplaces direct requests and manual API handling with the tideway Python package for querying active discovery runs. Updates initialization, search logic, and result processing to use tideway's appliance and data interfaces, simplifying code and improving reliability.* Refactor query loading and improve data handlingReplaced hardcoded query string with dynamic XML-based query loading using ElementTree. Added column renaming for consistency, expanded numeric column conversion, and updated code cells to reflect execution. These changes improve maintainability and data processing for active scan analysis.* Add query templates and clean active_scans notebookRemoved all content from notebooks/active_scans.ipynb, effectively clearing the notebook. Added new template notebooks for CSV and Tideway queries in notebooks/templates/csv_query_template.ipynb and notebooks/templates/tideway_query_template.ipynb., GitHub
v0.3.5 (#278)* 0.3.5Minor changes.* 0.3.6Small update.* 0.3.7-add-queries-xmlAdded an XML file with queries which can be used for direct import into Discovery.* Add new queries for credentials, devices, and scan rangesIntroduces multiple queries to capture unsupported device candidates, credential success/failure statistics (7 days and all time), session outpost mappings, DeviceInfo success counts, scheduled scan ranges, and exclude range definitions. These additions enhance reporting and analysis capabilities for device discovery and credential usage.* Add new queries for range, modules, agents, and errorsIntroduces several new queries: Exclude Ranges Detail, Knowledge Upload Modules, Host Utilisation, Installed Agents (Windows), and ECA Errors. Also improves formatting for the Discovery Run Analysis query for better readability.* Add KPI and discovery count queriesAdded multiple queries for counting discovered nodes by type (hosts, devices, software, cloud resources, etc.), schedule counts, unscanned connections, and licensing-related host tracking. These queries support KPI baseline reporting and improved discovery coverage analysis.* Add lifecycle report queries for DB, OS, and softwareIntroduces three new queries: Database Lifecycle Report, OS Lifecycle Report, and Software Lifecycle Report. These queries provide lifecycle dates and risk classifications for database software instances, host operating systems, and general software instances, supporting improved visibility into end-of-life and support status.* Add queries for removal candidates and orphaned VMsIntroduces new queries to identify near removal CI candidates, orphaned VMs, pattern modules, recently removed CIs, and missing VMs. Also improves formatting for existing scheduled and exclude range queries for better readability.* Add new queries for software users and SNMP devicesIntroduces a query for listing software instances with discovered user accounts and another for identifying unrecognised SNMP devices with successful sessions but no inferred host. These additions enhance visibility into user accounts on software instances and unsupported SNMP device discoveries.* Escape less-than signs in lifecycle queriesReplaced all '<' comparisons with '<' in lifecycle risk and report queries to ensure proper XML parsing and display. This change prevents syntax errors and improves compatibility with XML parsers.* Refactor active scans notebook for multi-appliance supportReorganized the notebook to support multiple BMC Discovery appliances, modularized configuration and session setup, and added functions for fetching and processing runs from each instance. Output CSVs are now saved per appliance, and code is more robust for error handling and extensibility.* Remove config.yaml loading and appliance selection codeDeleted a code cell that handled locating config.yaml, loading configuration, selecting an appliance, resolving API token, and preparing output directory. This streamlines the notebook and removes redundant or unnecessary setup logic., GitHub
v0.3.4 (#277)* Added new notebooksAdded notebooks for db_lifecycle, device_ids, devices and discovery_analysis.* More notebooksCreated additional notebooks covering excavation reports.* More notebooks* Final set of notebooks.* Schedule updaterAdded a schedule updater notebook.* git push –all –forcedevice_ids efficiencyUpdated device_Ids notebook to be more efficient and use pandas to wrangle exported data.* Added NB clear-nb script* updated discovery_analysis* v0.1.8Added KPI and Licensing dashboard (notebook)* v0.1.9Have fixed the queries and updated charts to show license threshold value.* v0.2.0Corrected Containers/Pods counts.* v0.2.1Combined lifecycle reports into one notebook.* v0.2.2Combined lifecycle reports into one notebook (missing from last push).* v0.2.3Updated appliance indexes.* v0.2.4Merge outputs fix so that all rows are copied (discovery handles unique).* 0.2.5Updated clear-nbs script to efficiently use a backup folder.Refactored capture candidates nb.* 0.2.5Completed refactor of capture candidates – leaner code.* 0.2.6Some refactoring.* 0.2.7mid-refacturing.* 0.2.8Completed refactor of credential_success report.* 0.2.9Refactors notebooks with more efficient functions.* 0.3.0Refactored IP analysis nb.* 0.3.1Updated ip address lookup and licensing threshold chart.* 0.3.2Refactored lifecycle reports.* 0.3.3Corrected save file locations.* 0.3.4Refactored various other single report notebooks., GitHub
v0.1.7 (#276)* Rename suggested cred opt report key* Expand expected report list and add Guide sheet* chore: clarify expected report descriptions* Fix report key membership check for cover sheet* docs: update report naming* Insert discovery instance column in capture candidates* feat: support YAML config and multiple appliances* Add configuration template and docs* Expand appliance config entries* Add config defaults for token and password files* Test appliance list credential overrides* Add fixture and tests for config appliances* Expose outpost credential details and merge into success report* Add outpost credential mapping helper and update reporting* Add outpost credentials report and CLI dispatch* Fix outpost credential map invocation* Handle missing appliance attribute in outpost creds* fix: avoid duplicate cred opt data* Merge fields from latest discovery record* Pass discovery data to output report* Add expected agent analysis* Fix outpost credential mapping* Limit discovery analysis to explicit request* Merge discovery records by iterating over history* Include consistency in dropped discovery records* Handle result variations in ip analysis* Handle empty report data* Add discovery run analysis report* refactor header handling* Use original query headers in tests and code* refactor: standardize query aliases* Remove unused CamelCase normalization* Use raw headers in CSV exports* Use endpoint field in unique_identities and expand tests* Add test for shared device identities* Add test to ensure queries import without warnings* Handle invalid orphan VM responses* Log devices test output to file* Remove obsolete slash stripping in tku test* docs: add AGENTS instructions for junior-friendly code* Handle API timeouts gracefully and fix debug config* Add raw query execution option* Add active_runs query and document query option* Run all report queries with –queries* refactor active_runs query* Add query duplication tests and sanitize queries* Remove __had_inference filter from DeviceInfo success queries* Handle prefixed credential paths in session lookups* Ensure failure-only creds show 7-day zeros* Normalize credential IDs and map outposts* Fix credential success CLI header parsing* Rename active runs report to active scans* Rename active scans output* Add UUID column to query results* refactor success report totals* Add uuid fallback for session results* Show zero counts for inactive credentials* Update reporting.pyAdded some debugging info.* Refactor debug logging* Normalize credential UUID handling* Add debug logging for activity and 7-day success metrics* fix credential success active status and add outpost* Handle dict-wrapped results in session_get* Test successful handles dict results* Log payload for unsupported session_get types* Handle explicit zero counts as active* Normalize nested search results tables* Handle table-style inputs in session_get* feat: paginate search results and extend credential reporting* Warn on server errors in search_results* Add search result caching* Add max thread option for reporting* Add persistent cache for API queries and offline reporting* Handle server-side pagination in search_results* Handle over-limit search responses* fix endless pagination loop* cache unique identities* feat: limit unique identities* Remove duplicate overfetch check* Remove unused discovery access lookup in unique_identities* fix: escape rsyslog sed command* feat: retry search on gateway timeouts* Run device_ids query with –queries* Add DeviceInfo base query* Add deviceInfo_access query* Add DeviceInfo network query* feat: split device lookup queries* Refactor device_ids report to use base queries* Refine device identity queries* feat: show progress for unique identities* Remove coverage from device_ids report* Use unified device identity query* Add devices report query mapping and tests* Refactor device report to use devices_lookup* Split deviceInfo run_queries into base access network* Test coverage for devices lookup* Revert "Refactor device report to use devices_lookup"* Split device report into granular queries* Ensure device last fields updated without marker* Ensure devices report loads full dataset* Fixed post processing line.Removed 'endpoint' attribute.* Populate discovery analysis device fields* Add chunked search helper and wrapper* feat: allow tuning search page size* Refactor discovery queries to use incremental detail fetching* Add export search and CLI option* Chunk last disco query* Split last_disco into chunked queries* refactor discovery data gathering to use chunked queries* Update reporting tests for chunked discovery query* Handle repeated API timeouts* Split discovery access key queries and handle partial timeouts* v0.1.7Added Preserve-Existing SupportAdded additional Timeout HandlingImprovements to Output/FormattingAdded Notebooks for troubleshooting and testing adjustments to queries, GitHub
Merge pull request #42 from codefitz/02wwd6-codex/add-function-to-retrieve-outpost-urlsAdd outpost URL mapping to credential success, GitHub

Share this:

Related

Discover more from fitzmoskal