Closed ATM View - 24ACMA050
To establish Contract for Analysis of metrics to assess digital platform effectiveness under voluntary, self-regulatory
Canberra, Sydney, Melbourne, Adelaide, Perth, Brisbane, Darwin, Hobart
Background
In Australia, minimising the risk of harm from mis- and disinformation on digital platforms has been the subject of self-regulation by the digital platforms industry since 2021 through the (the code). This approach recognises that digital platforms must be responsible and accountable for the content they allow to appear and spread on their platforms, while balancing freedom of expression and other important rights.
The code (developed by the Digital Industry Group Inc. (DIGI) at the request of the Australian Government) requires signatories to identify, assess and address mis- and disinformation on their services. outcomes-based. Each signatory agrees to enact its own measures to achieve the codes’ objectives (whether mandatory or those it has opted-into). TX Corp was an original signatory to the code but had its signatory status withdrawn in November 2023 after a complaints process under the code found that it failed to provide a mechanism to the public to make reports of breaches of its policies for an extended period.
The Customer is the Australian independent statutory authority that administers Australia’s telecommunications, broadcasting and some online regulation, including the development of codes of practice.
In 2019, the Customer was tasked by the Australian Government with overseeing the development of the code and we continue to perform an oversight role. As part of this oversight, we have provided the Australian Government with two reports on the In our , the Customer encouraged Snap, Advance Publications (Reddit) and Spotify to sign up to the code as platforms with large Australian active user bases.
Among other things, in reporting on the code’s adequacy, the Customer has emphasised that to be effective, it must be supported by robust measurement and performance reporting frameworks that provide sufficient levels of accountability and public transparency. The Customer’s views on the importance of measurement and performance for the code are also consistent with the experience of the European Union’s (EU) Code of Practice on Disinformation. Under Commitment 41 of the EU Code, signatories have committed to developing ‘structural indicators’ designed to assess the effectiveness of the code in reducing the spread of online disinformation for each relevant signatory and for the entire online ecosystem in the EU and at the Member State level.
·
·
The Customer will consult with DIGI, signatories, other interested digital platforms and experts from across academia, government and industry to finalise the framework. The successful tenderer will be provided with a copy of the draft framework, noting the framework may include up to 67 metrics.
The Customer has previously indicated that the kinds of metrics appropriately included in a measurement framework include:
· Data on user reports or user-flagged instances of potentially harmful misinformation targeting Australians on the platform.
· Data on human-moderated or algorithmically identified instances of potentially harmful misinformation targeting Australians on the platform.
· Data on the actions taken by the platform to address potentially harmful misinformation once identified (broken down by category such as take-downs and credibility signalling).
· Data on the effectiveness of these actions and measures in reducing the volume and impact of potentially harmful misinformation on their platform.
Required Services
The Customer is seeking tenders from suitably qualified Potential Suppliers to provide Part 1, Part 2 or both parts outlined in this section. Potential Suppliers are asked to quote separately against Parts 1 and 2, as well outlined any proposed discounts or efficiencies if both parts are contracted. Where potential Suppliers have capabilities against either Part 1 or Part 2, they are encouraged to quote for that relevant Part.
PART 1
We are seeking a Supplier to provide independent expert advice to validate the metrics (as opposed to measures) which we have identified in the draft framework for the code’s current 8 signatories as well as other non-signatories, specifically X Corp (formerly Twitter), Snap, Advance Publications (Reddit) and Spotify.
The Customer will provide the successful Supplier with the draft measurement framework, which may include up to 67 quantitative metrics. The Supplier will not have a role in consultations on the framework.
The Supplier will be required to review the Customer’s proposed metrics and perform a validation check of their feasibility and utility. As part of this validation check, we are seeking to understand:
1. whether each digital platform (12 platforms total) is likely to be able to provide data against each relevant metric
2. whether the proposed metrics are likely to provide Australian-specific insights into the success of the measures that signatories have implemented to meet the outcomes of Objectives 1 and 3 of the code.
3. a recommendation/view on whether alternative metrics would yield better Australian-specific insights into the success of the measures to meet the outcomes of Objectives 1 and 3 of the code, and if so, what those alternative metrics would be
4. a recommendation/view on whether and/or how information reported by signatories against these metrics could be independently verified
5. a recommendation/view on any other relevant issues and/ or barriers to implementation of the framework.
In tendering for Part 1, Potential Suppliers must outline their proposed approach to the validation exercise, including proposed methodology and proposed timeline for completion of key deliverables. Potential Suppliers must also provide details of relevant personnel and their expertise and experience, as well as declare any potential conflicts of interest.
Key deliverables under Part 1 are:
· A finalised scoping document: Following consultation with the Customer the Supplier must provide a final scoping document by 10 April 2024, setting out its final project plan, timeline and proposed methodology following initial review of the draft measurement framework. Any significant changes must be discussed with the Customer and the Customer will approve the final scoping document before further work commences.
· A draft report: the Supplier will provide the Customer with its draft final report, and the Customer will approve the draft report before a final report is delivered. The Customer is open to the final report being delivered in different formats.
· Other deliverables as stipulated in its scoping document: Where the Supplier has stipulated other key deliverables in its scoping document, it must provide these by agreed dates. The Customer will approve these deliverables before a final report is accepted.
· A final report: The Supplier will provide a final report to the Customer's satisfaction by
24 June 2024. The report must be compliant with WCAG2.0 where relevant.
PART 2
We are seeking a Supplier to provide data (where available) against a finalised set of metrics for the code’s current 8 signatories as well as other non-signatories, specifically X Corp (formerly Twitter), Snap, Advance Publications (Reddit) and Spotify. The finalised metrics will be provided by the Customer following consideration of the consultation process on the draft framework and/or advice provided in response to Part 1 (if the tender for Part 1 is accepted).
This data is intended to form a set of baseline data to be used by the Customer to compare future reports provided by signatories and other digital platforms.
In tendering for Part 2, the Potential Suppliers must provide a draft scoping document, setting out its project plan, proposed timeline and proposed methodology for obtaining the data. The Supplier should also provide information on the relevant personnel to conduct the work including their expertise and experience, and any potential conflicts of interest.
The Customer is open to receiving these deliverables either as combined documents with Part 1 deliverables (i.e. a single scoping document comprising of 2 parts and a timeline including key milestones for both parts) or as separate deliverables.
Key deliverables under Part 2 are:
· A finalised scoping document: Following consultation with the Customer the Supplier must provide a final scoping document to the Customer by 10 April 2024 setting out its final project plan, timeline and methodology for obtaining the data. Any significant changes must be discussed with the Customer and the Customer will approve the final scoping document before further work commences.
No conditions for participation are specified.
4 Months