Link tags: testing

Hallucinations in code are the least dangerous form of LLM mistakes

The moment you run LLM generated code, any hallucinated methods will be instantly obvious: you’ll get an error. You can fix that yourself or you can feed the error back into the LLM and watch it correct itself.

Compare this to hallucinations in regular prose, where you need a critical eye, strong intuitions and well developed fact checking skills to avoid sharing information that’s incorrect and directly harmful to your reputation.

With code you get a powerful form of fact checking for free. Run the code, see if it works.

Moving on from React, a Year Later

Many interactions are not possible without JavaScript, but that doesn’t mean we should look to write more than we have to. The server doing something useful is a requirement for building an interesting business. The client doing something is often a nice-to-have.

There’s also this:

It’s really fast

One of the arguments for a SPA is that it provides a more reactive customer experience. I think that’s mostly debunked at this point, due to the performance creep and complexity that comes in with a more complicated client-server relationship.

Website Speed Test

Here’s a handy free tool from Calibre that’ll give your website a performance assessment.

80 / 20 accessibility · marcus.io

So my observation is that 80% of the subject of accessibility consists of fairly simple basics that can probably be learnt in 20% of the time available. The remaining 20% are the difficult situations, edge cases, assistive technology support gaps and corners of specialised knowledge, but these are extrapolated to 100% of the subject, giving it a bad, anxiety-inducing and difficult reputation overall.

Home - Sa11y

Another handy accessibility testing tool that can be used as a bookmarklet.

Bugs I’ve filed on browsers | Read the Tea Leaves

I think filing bugs on browsers is one of the most useful things a web developer can do.

Agreed!

Accessibility Personas

Personas are often toothless, but these accessibility personas from gov.uk are more practical and useful than most:

Each profile has a different simulation of their persona’s condition and runs the assistive technology they use to help them.

You can use these profiles to experience the web from the perspective of the personas and gain more understanding of accessibility issues.

Improving the information architecture of the Smart Pension member app | Design and tech | Smart – retirement, savings and financial wellbeing

Here’s a really excellent, clearly-written case study that unfortunately includes this accurate observation:

In recent years the practice of information architecture has fallen out of fashion, which is a shame as you can’t design something successfully without it. If a user can’t find a feature, it’s game over - the feature may as well not exist as far as they’re concerned.

I also like this insight:

Burger menus are effective… at hiding things.

Introducing Opportunities & Experiments: Taking the Guesswork out of Performance - WebPageTest Blog

WebPageTest just got even better! Now you can mimic the results of what would’ve previously required actually shipping, like adding third-party scripts, switching from a client-rendered to a server-rendered architecture and other changes that could potentially have a big effect on performance. Now you can run an experiment to get the results before actual implementation.

Paper Prototype CSS

A stylesheet to imitate paper—perfect for low-fidelity prototypes that you want to test.

CSS { In Real Life } | My Browser Support Strategy

This is a great succinct definition of progressive enhancement:

Progressive enhancement is a web development strategy by which we ensure that the essential content and functionality of a website is accessible to as many users as possible, while providing an improved experience using newer features for users whose devices are capable of supporting them.

The monoculture web

Firefox as the asphyxiating canary in the coalmine of the web.

My Challenge to the Web Performance Community — Philip Walton

I’ve noticed a trend in recent years—a trend that I’ve admittedly been part of myself—where performance-minded developers will rebuild a site and then post a screenshot of their Lighthouse score on social media to show off how fast it is.

Mea culpa! I should post my CrUX reports too.

But I’m going to respectfully decline Phil’s advice to use any of the RUM analytics providers he recommends that require me to put another script element on my site. One third-party script is one third-party script too many.

Benjamin Parry~ Writing ~ Engineering a better design test ~ @benjaminparry

It sometimes feels like we end up testing the limitations of our tools rather than the content and design itself.

What Benjamin found—and I heartily agree—is that HTML prototypes give you the most bang for your buck:

At the point of preparing for usability testing, it seemed ludicrous to move to any prototyping material other than the one we were already building in. The bedrock of the web: HTML, CSS and Javascript.

A/A Testing: How I increased conversions 300% by doing absolutely nothing

An oldie but a goodie. If you think you’re getting statistically significant results from A/B testing, you should probably consider doing some A/A testing.

In an A/A test, you run a test using the exact same options for both “variants” in your test.

Building a resilient frontend using progressive enhancement - Service Manual - GOV.UK

Using progressive enhancement means your users will be able to do what they need to do if any part of the stack fails.

What a terrific short guide to sensible web development!

• Start with HTML
  • Using interactive elements
  • Adding the extras
  • Building more complex services
  • Testing your service
  • Do not assume users turn off CSS or JavaScript
  • Case studies and related guides

Performance-testing the Google I/O site - JakeArchibald.com

Modern web development:

Imagine you went to a restaurant, took a seat, and 20 minutes later you still haven’t been given a menu. You ask where it is, and you’re told “oh, we’re currently cooking you everything you might possibly ask for. Then we’ll give you the menu, you’ll pick something, and we’ll be able to give you it instantly, because it’ll all be ready”.

Single page apps, ladies and gentlemen.

Global performance insights for your site | Lighthouse Metrics

I hadn’t come across this before—run Lighthouse tests on your pages from six different locations around the world at once.

How I Build JavaScript Apps In 2021

Sensible:

• I avoid build processes.
• I avoid transpiling.
• I avoid the new and shiny.
• I test EvErYtHiNg.
• I optimize for performance and quality.
• I use my own work.
• I use open source to my advantage.

An opinionated guide to accessibility testing /// Iain Bean

0. First impressions
1. The Tab key
2. Automated testing tools
3. Screen reader testing
4. Next steps