ok, clear
The only issue is that if something like that AI summary is posted here, without the original executive summary for context, people will just look at the AI (as many do for Google searches now) and take that as Gospel
CityCyclingEdinburgh Forum » Infrastructure
Roseburn to Leith consultation begins (and the debate continues!) CCWEL
(5587 posts)-
Posted 1 day ago #
-
I know there is a huge push within the public sector to use AI to analyse consultation responses, as it is difficult to do well, requires highly trained staff, takes ages, costs a fortune and the end users (senior managers and elected members) typically have no idea of why any of this is the case and simply see it as a drain on resources - weirdly, not running a consultation is never seen as the solution to this.
Public sector managers, always suckers for tech hucksters that promise efficiency savings, are likely to start trialling this soon (public sector trials or pilots almost never see something being rejected as a bad idea as it reflects badly on whoever proposed it).
This could be bad for 'us' as AI analysis is more likely to treat consultations as referenda, rather than discerning between BS and valid/on topic responses.
Posted 1 day ago # -
For what it's worth, I believe that summarising text is one of the things AI is reasonable at in terms of lower hallucination count? However, ironically I might be hallucinating that I read that somewhere.
Posted 1 day ago # -
@bakky - I bow to your superior judgement on technical issues. However, my concern is that the analysis of consultation responses is simply seen as a summarising exercise, or analysing 'sentiment'.
The real human skill is understanding the context, going beyond analysing 'sentiment' to understanding whether the thinking behind that sentiment is based on a reasonable interpretation of fact.
I suspect AI would be quite good at coding responses and identifying hot topics, which can look very much like an actual analysis, without providing much in the way of true insight. This type of pseudo-analysis will probably be enough to convince most public sector managers mind.
Posted 1 day ago # -
You can get whatever kind of analysis you want out of it; if you prime it to be suspicious of NIMBY responses that might be using a legitimate sounding concern to block road space reallocation that's what it will give you. I don't think you could just ask it to analyse responses without providing any guidance on how to intepret them, or each model generation might give you different results.
For example when I asked chatGPT to assess a certain charity's consultation response on Spaces for People, it said:
It briefly acknowledges pandemic distancing aims and notes potential upsides like wider pavements and reduced clutter, but provides no discussion or evidence of the well-documented safety gains of protected cycleways for children/families (e.g., mode shift, collision reduction, independent travel).
The core of the piece is a catalogue of accessibility risks (floating bus stops, loss of kerb access/Blue Badge bays, confusing temporary layouts, inaccessible consultations), survey stats highlighting disabled users’ difficulties, and a call to halt permanence without fuller Equality Impact Assessments and better consultation.
The only “benefits” retained are generic (wider pavements), with no attempt to balance them against child safety outcomes or to explore mitigations that reconcile both (e.g., accessible bus stop designs, tactile delineation, controlled crossings).
Bottom line: it’s a focused, one-sided critique aimed at ensuring accessibility and due process; it does not fairly weigh or even substantively acknowledge the dramatic safety benefits for children.
but if you ask it the same question with different framing, this is
"a vulnerable group whose independence and safety are compromised, and it makes a strong case that their needs have been overlooked in the rush to install active travel infrastructure."
Posted 1 day ago # -
So
AI will be useful when it realises it’s being asked leading questions?
Or not…
Posted 1 day ago # -
@Dave - thanks, that's fairly eye opening. Could something similar apply to the analysis and reporting on hundreds, or even thousands, of responses to a single consultation?
Posted 1 day ago #
Reply
You must log in to post.
posts
