Closed

Delivery of Data Science, Analytics, and Machine Learning Governance Framework

Tender ID: 533946


Tender Details

Tender #:
DM-19623  
Status:
Closed
Publish Date:
10 August 2023
Closing Date:
21 August 2023

Tender Description

This Tender is invited by the Issuer.

At Jobs and Skills Australia, we are striving to make engaging with us easily by facilitating our engagement in ways that meet the needs of our partners and stakeholders. This will inform our communications and consultation strategies and shape our digital and data platform design.

The JSA currently consists of approximately 220 staff (including contractors), with a large volume of these staff in data roles spread across five branches. Approximately 10% of staff are data scientists spread across multiple teams.

An Azure cloud platform for data analytics has been established and is comprised of Azure Data Lake Service, Azure Data Factory, DevOps, and Databricks. A Posit Teams platform has also been deployed on Linux on-prem infrastructure, with a primary purpose being the development and publication of complex data visualisation, consuming data that has been curated in the JSA Azure environment. The uptake of these platforms has been organic within a relatively small number of teams in JSA who have their own informal models of technical governance. As the JSA data platforms mature, we are observing the uptake of the newer technologies becoming more widespread across JSA for both new projects and the transition of legacy work. JSA also has a volume of work that is performed using more traditional tooling such as SAS, Power BI, Qlik, Enterprise Data Warehousing, and Microsoft Excel.

We are seeking a seller with expertise in establishing governance frameworks and processes using CI/CD methodologies and principles for data science, data analytics, and machine learning models. The seller will be required to:

  • have a solid understanding of CI/CD methodologies (preferably in Azure and Databricks).
  • review existing informal governance measures in place within JSA teams.
  • review existing code and products.
  • undertake an inclusive consultation process.
  • deliver an agency-wide technical governance framework for implementation.

The framework should be principles-based, with the flexibility to deliver good governance outcomes in a variety of ways. In addition, the seller will be required to deliver process documentation, including templates and guidance material, for data science, analytics, and machine learning models developed on the JSA’s Azure and Posit platforms.