Tags: bias

The Jeopardy Phenomenon – Chris Coyier

AI has the Jeopardy Phenomenon too.

If you use it to generate code that is outside your expertise, you are likely to think it’s all well and good, especially if it seems to work at first pop. But if you’re intimately familiar with the technology or the code around the code it’s generating, there is a good chance you’ll be like hey! that’s not quite right!

Not just code. I’m astounded by the cognitive dissonance displayed by people who say “I asked an LLM about {topic I’m familiar with}, and here’s all the things it got wrong” who then proceed to say “It was really useful when I asked an LLM for advice on {topic I’m not familiar with, hence why I’m asking an LLM for advice}.”

Like, if you know that the results are super dodgy for your own area of expertise, why would you think they’d be any better for, I don’t know, restaurant recommendations in a city you’ve never been to?

Stop saying that AI is just a tool and it only matters how it is used | Frank Elavsky

Must be something in the air.

One persistent piece of slopaganda you’ll hear is this:

“It’s just a tool. What matters is how you use it.”

This isn’t a new tack. The same justification has been applied to many technologies.

Leaving aside Kranzberg’s first law, large language models are the very antithesis of a neutral technology. They’re imbued with bias and political decisions at every level.

There’s the obvious problem of where the training data comes from. It’s stolen. Everyone knows this, but some people would rather pretend they don’t know how the sausage is made.

But if you set aside how the tool is made, it’s still just a tool, right? A building is still a building even if it’s built on stolen land.

Except with large language models, the training data is just the first step. After that you need to traumatise an underpaid workforce to remove the most horrifying content. Then you build an opaque black box that end-users have no control over.

Take temperature, for example. That’s the degree of probability a large language model uses for choosing the next token. Dial the temperature too low and the tool will parrot its training data too closely, making it a plagiarism machine. Dial the temperature too high and the tool generates what we kindly call “hallucinations”.

Either way, you have no control over that dial. Someone else is making that decision for you.

A large language model is as neutral as an AK-47.

I understand why people want to feel in control of the tools they’re using. I know why people will use large language models for some tasks—brainstorming, rubber ducking—but strictly avoid them for any outputs intended for human consumption.

You could even convince yourself that a large language model is like a bicycle for the mind. In truth, a large language model is more like one of those hover chairs on the spaceship in WALL·E.

Large language models don’t amplify your creativity and agency. Large language models stunt your creativity and rob you of agency.

When someone applies a large language model it is an example of tool use. But the large language model isn’t the tool.

Bias in Design Systems - bencallahan.com

Thoughtful analysis from Ben (as always).

Another uncalled-for blog post about the ethics of using AI | Clagnut by Richard Rutter

This is a really thoughtful piece by Rich, who’s got conflicted feelings about large language models in the design process. I suspect a lot of people can relate to this.

What I do know is that I find LLMs useful on occasion, but every time I use one I die a little inside.

There are plenty of non-men in tech to speak at your conference - Stephanie Stimac’s Blog

I’m going to share this with the organisers of that conference I pulled out of recently.

When I was looking back at 2024, I mentioned that I didn’t give a single conference talk (though I did host three conferences—Patterns Day, CSS Day, and UX London).

I almost spoke at a conference though. I was all set to speak at an event in the Netherlands. But then the line-up was announced and I was kind of shocked at the lack of representation. The schedule was dominated by white dudes like me. There were just four women in a line-up of 30 speakers.

When I raised my concerns, I was told:

We did receive a lot of talks, but almost no women because there are almost no women in this kind of jobs.

Yikes! I withdrew my participation.

I wish I could say that it was one-off occurrence, but it just happened again.

I was looking forward to speaking at DevDays Europe. I’ve never been to Vilnius but I’ve heard it’s lovely.

Now, to be fair, I don’t think the line-up is finalised, but it’s not looking good.

Once again, I raised my concerns. I was told:

Unfortunately, we do not get a lot of applications from women and have to work with what we have.

Even though I knew I was just proving Brandolini’s law, I tried to point out the problems with that attitude (while also explaining that I’ve curated many confernce line-ups myself):

It’s not really conference curation if you rely purely on whoever happens to submit a proposal. Surely you must accept some responsibility for ensuring a good diverse line-up?

The response began with:

I agree that it’s important to address the lack of diversity.

…but then went on:

I just wanted to share that the developer field as a whole tends to be male-dominated, not just among speakers but also attendees.

At this point, I’m face-palming. I tried pointing out that there might just be a connection between the make-up of the attendees and the make-up of the speaker line-up. Heck, if I feel uncomfortable attending such a homogeneous conference, imagine what a woman developer would think!

Then they dropped the real clanger:

While we always aim for a diverse line-up, our main focus has been on ensuring high-quality presentations and providing the best experience for our audience.

Double-yikes! I tried to remain calm in my response. I asked them to stop and think about what they were implying. They’re literally setting up a dichotomy between having a diverse line-up and having a good line-up. Like it’s inconceivable you could have both. As though one must come at the expense of the other. Just think about the deeply embedded bias that would enable that kind of worldview.

Needless to say, I won’t be speaking at that event.

This is depressing. It feels like we’re backsliding to what conferences were like 15 years ago.

I can’t help but spot the commonalaties between the offending events. Both of them have multiple tracks. Both of them have a policy of not paying their speakers. Both of them seem to think that opening up a form for people to submit proposals counts as curation. It doesn’t.

Don’t get me wrong. Having a call for proposals is great …as long as it’s part of an overall curation strategy that actually values diversity.

You can submit a proposal to speak at FFconf, for example. But Remy doesn’t limit his options to what people submit. He puts a lot of work into creating a superb line-up that is always diverse, and always excellent.

By the way, you can also submit a proposal for UX London. I’ve had lots of submissions so far, but again, I’m not going to limit my pool of potential speakers to just the people who know about that application form. That would be a classic example of the streetlight effect:

The streetlight effect, or the drunkard’s search principle, is a type of observational bias that occurs when people only search for something where it is easiest to look.

It’s quite depressing to see this kind of minimal-viable conference curation result in such heavily skewed line-ups. Withdrawing from speaking at those events is literally the least I can do.

I’m with Karolina:

What I’m looking for: at least 40% of speakers have to be women speaking on the subject of their expertise instead of being invited to present for the sake of adjusting the conference quotas. I want to see people of colour too. In an ideal scenario, I’d like to see as many gender identities, ethnical backgrounds, ages and races as possible.

Artificial intelligence: who owns the future? - ethical.net

Whether consciously or not, AI manufacturers have decided to prioritise plausibility over accuracy. It means AI systems are impressive, but in a world plagued by conspiracy and disinformation this decision only deepens the problem.

React Bias

Dev perception.

The juxtaposition of The HTTP Archive’s analysis and The State of JS 2020 Survey results suggest that a disproportionately small—yet exceedingly vocal minority—of white male developers advocate strongly for React, and by extension, a development experience that favors thick client/thin server architectures which are given to poor performance in adverse conditions. Such conditions are less likely to be experienced by white male developers themselves, therefore reaffirming and reflecting their own biases in their work.

Coded Bias Official Trailer on Vimeo

Coded Bias follows MIT Media Lab researcher Joy Buolamwini’s startling discovery that many facial recognition technologies fail more often on darker-skinned faces, and delves into an investigation of widespread bias in artificial intelligence.

Artificial Intelligence: Threat or Menace? - Charlie’s Diary

I am not a believer in the AI singularity — the rapture of the nerds — that is, in the possibility of building a brain-in-a-box that will self-improve its own capabilities until it outstrips our ability to keep up. What CS professor and fellow SF author Vernor Vinge described as “the last invention humans will ever need to make”. But I do think we’re going to keep building more and more complicated, systems that are opaque rather than transparent, and that launder our unspoken prejudices and encode them in our social environment. As our widely-deployed neural processors get more powerful, the decisions they take will become harder and harder to question or oppose. And that’s the real threat of AI — not killer robots, but “computer says no” without recourse to appeal.

Untold History of AI - IEEE Spectrum

A terrific six-part series of short articles looking at the people behind the history of Artificial Intelligence, from Babbage to Turing to JCR Licklider.

1. When Charles Babbage Played Chess With the Original Mechanical Turk
2. Invisible Women Programmed America’s First Electronic Computer
3. Why Alan Turing Wanted AI Agents to Make Mistakes
4. The DARPA Dreamer Who Aimed for Cyborg Intelligence
5. Algorithmic Bias Was Born in the 1980s
6. How Amazon’s Mechanical Turkers Got Squeezed Inside the Machine

The history of AI is often told as the story of machines getting smarter over time. What’s lost is the human element in the narrative, how intelligent machines are designed, trained, and powered by human minds and bodies.

Programming as translation – Increment: Internationalization

Programming lessons from Umberto Eco and Emily Wilson.

Converting the analog into the digital requires discretization, leaving things out. What we filter out—or what we focus on—depends on our biases. How do conventional translators handle issues of bias? What can programmers learn from them?

Why Data Is Never Raw - The New Atlantis

Raw data is both an oxymoron and a bad idea; to the contrary, data should be cooked with care.

Big ol’ Ball o’ JavaScript | Brad Frost

Backend logic? JavaScript. Styles? We do that in JavaScript now. Markup? JavaScript. Anything else? JavaScript.

Historically, different languages suggested different roles. “This language does style.” “This language does structure.” But now it’s “This JavaScript does style.” “This JavaScript does structure.” “This JavaScript does database queries.”

Reluctant Gatekeeping: The Problem With Full Stack | HeydonWorks

The value you want form a CSS expert is their CSS, not their JavaScript, so it’s absurd to make JavaScript a requirement.

Absolutely spot on! And it cuts both ways:

Put CSS in JS and anyone who wishes to write CSS now has to know JavaScript. Not just JavaScript, but —most likely—the specific ‘flavor’ of JavaScript called React. That’s gatekeeping, first of all, but the worst part is the JavaScript aficionado didn’t want CSS on their plate in the first place.

[Essay] Known Unknowns | New Dark Age by James Bridle | Harper’s Magazine

A terrific cautionary look at the history of machine learning and artificial intelligence from the new laugh-a-minute book by James.

Artificial Intelligence for more human interfaces | Christian Heilmann

An even-handed assessment of the benefits and dangers of machine learning.

What does sponsorship look like? | Lara Hogan

It’s upsetting to realize that the reason why you’re in a senior position may be because of the system of privilege that got you there. It’s upsetting to realize that there are people who aren’t in that rank who are more qualified than you, but who haven’t benefited from the same privilege you did.

It me.

So here’s what I can do about it:

• Start sponsoring members of underrepresented groups
• Listen to marginalized people, and believe them
• Do “the homework” to be a better mentor

Future Ethics

Cennydd is writing (and self-publishing) a book on ethics and digital design. It will be released in September.

Technology is never neutral: it has inevitable social, political, and moral impact. The coming era of connected smart technologies, such as AI, autonomous vehicles, and the Internet of Things, demands trust: trust the tech industry has yet to fully earn.