Automated accessibility testing at Slack
slack.engineering

・

130 points

・

teivah

・

3 days ago

58 comments

dbjorge ・ 3 days ago

I am one of the maintainers of the axe accessibility testing engine the Slack team is using. It's awesome to see such a detailed writeup of how folks are building on our team's work!

We publish the engine (axe-core) and the "core" playwright integration library Slack is using (@axe-core/playwright) as open source, but if you're interested in what the Slack team has described in this blog, we also have a paid offering called axe Developer Hub (https://www.deque.com/axe/developer-hub) that offers a similar workflow to what the Slack folks describe here: It hooks into end-to-end tests you already have to add in accessibility testing without needing a ton of code changes to your test suite.

It's very enlightening to see which features the Slack folks prioritized for their setup and to see some of the stuff they were able to do by going deep on integration with Playwright specifically. It's not often you are lucky enough to get feedback as strong as "we cared about <feature> enough to invest a bunch of engineering time into it".

If you're interested in building these sort of accessibility tools, my team is hiring! https://www.deque.com/careers/senior-accessibility-tool-deve...

  • rasjani ・ 3 days ago

    Been running somewhat similar combo few months and toy'd around with taking the screenshots of the selectors axe-core/playwright reports as I didn't notice such feature in neither playwright or its html reporter. Do you happen to know if slack patched this or is this feature available somehow ?

    And if you are willing to answer some other questions regarding axe-core itself, I might have few.

    • dbjorge ・ 3 days ago

      Screenshots of issues is something we support in many of our paid offerings, but not in axe-core itself. It sounds like the Slack folks implemented their own version of it.

      If you have general questions about axe-core, the best place to ask is our axe Community slack instance (https://accessibility.deque.com/axe-community). If you have a specific issue you'd like us to investigate, try https://github.com/dequelabs/axe-core/issues

      • rasjani ・ 2 days ago

        Either there's some very long queue or that signup page for slack invite doesn't work.

skeptrune ・ 3 days ago

>A few developers gave us early feedback and requested screenshots of the pages where accessibility violations occurred.

It's amazing how much a screenshot will do for my motivation to fix a frontend bug. Visually identifying severity is much easier than reading and making a mental judgement.

  • steve_adams_86 ・ 3 days ago

    I like using video as well. The whole "picture is worth 1000 words" adage is true, and it makes all kinds of bugs far easier to recognize.

    I'm sure other tools are great too but I find Cleanshot on macOS makes it super convenient to do it, so there's no excuse not to document reports with images and/or videos.

    I do the same with pull requests. Words are almost always essential, but demonstrating bugs/changes/features directly through accompanying visuals is hard to beat.

    • chatmasta ・ 3 days ago

      I use native Cmd+Shift+5 to record video and then convert it to .mp4 with ffmpeg -i bug-report.mov bug-report.mp4

    • PyWoody ・ 3 days ago

      If you don't need to edit the screenshot, Command+Control+Shift+4 will bring up crosshairs that will put the screenshot selection in your clipboard and won't save it to disk. It's super handy for doing a ton of quick, one-offs.

      • MontagFTB ・ 3 days ago
        ・ 2 more

        If it helps, hitting the spacebar will toggle screenshotting a window.

        • agos ・ 3 days ago

          and it includes the shadow, with trasparency! looks great

    • wccrawford ・ 3 days ago

      I dreaded videos for a few reasons, some of which could have been fixed, but some were human nature.

      First, they didn't play in all browsers at the company I worked at. That meant I had to either download it, or use a different browser for that.

      But even then, it was a game "guess what the CSR thought was wrong" in the video. Usually after watching a rather long intro sequence before getting to the actual bug.

      If the company is trying to replace written bug reports with videos for speed or convenience, it's a nightmare for the devs.

      If it's just an add-on to show the specifics, then it might actually be good. I rarely got those.

    • shafyy ・ 3 days ago

      I love Cleanshot!

butz ・ 3 days ago

Article about accessibility - code snippets added as images. And what about alt text? It just broadly summarizes what is displayed in image. I think there might be a tiny little problem here.

  • sunshowers ・ 3 days ago

    The text is also quite small and low-contrast (which matters a lot for accessibility!)

    Also, more subjectively, the snippets don't really match the aesthetics of the rest of the site. The pseudo-macOS rendering inside the black borders is strange, as is the choice to use different monospace fonts for filenames and code snippets.

  • geodel ・ 3 days ago

    The things you are asking would be done in 2028 under "Grounds up refactoring for Accessibility First Engineering"

  • superpie ・ 3 days ago

    [ Speaking Spanish ]

gostsamo ・ 3 days ago

Slack have a rather good a11y experience. The few bugs that I've reported had been addressed rather promptly. The ux is also good with obvious consideration given to keyboard navigation. There are a few annoyances which separate it from perfect, but I rather like the app, especially compared to teams.

  • password4321 ・ 3 days ago

    > I rather like the app, especially compared to teams

    Pretty sure just about every other option is pretty likeable compared to Teams!

    • eitally ・ 3 days ago

      Yet there are Teams features that nobody else has emulated yet and which quickly become almost indispensable for business use. I'll name two:

      1. Teams automatically creates a chat group for every Teams calendar event. This can include external attendees.

      2. Teams is useful for chat & meetings, but Teams spaces are hugely helpful as document repositories, too, and it's additionally easy to add things like Gantt charts and other enriched content types through add-ins.

      And a bonus one:

      3. The ability to seamlessly transfer a Teams meeting connection between arbitrary devices (laptop -> desktop, phone -> laptop, etc).

      • sofixa ・ 3 days ago

        > Teams automatically creates a chat group for every Teams calendar event. This can include external attendees

        Great, so each conversation is spread out and siloed. Thankfully Teams has good search and you can find stuff, right? Right?

        > Teams is useful for chat & meetings, but Teams spaces are hugely helpful as document repositories, too, and it's additionally easy to add things like Gantt charts and other enriched content types through add-ins

        Slack Canvas kind of does this. I'm not convinced having all your documentation in your chat/meeting app makes sense, but it could be useful.

        > The ability to seamlessly transfer a Teams meeting connection between arbitrary devices (laptop -> desktop, phone -> laptop, etc).

        Zoom does this too, and has for years. I think I've seen Slack huddles offer the same option too.

      • gostsamo ・ 3 days ago

        Additional feature of Teams is that their web app refuses to work on my firefox after working until two months ago. I'm not a heavy business user and the casual experience with this thing is pretty annoying.

      • robertlagrant ・ 3 days ago
        ・ 2 more

        > 1. Teams automatically creates a chat group for every Teams calendar event. This can include external attendees.

        This happens in Zoom as well, without the extra Teams downside of external attendees being 4th class citizens.

        > 2. Teams is useful for chat & meetings, but Teams spaces are hugely helpful as document repositories, too, and it's additionally easy to add things like Gantt charts and other enriched content types through add-ins.

        This is really simple in demos, but tends towards sharing documents via email again (for external attendees) or growing the channel size massively (for internals who need the documents but aren't obviously needed in the team).

        • eitally ・ 15 hours ago

          I does also help propagate the other "elephant in the room" problem M365 users face, which is document version control. Since people are constantly sharing actual files -- vs symlinks as with Google Workspace -- nobody ever really knows who has the canonical version of whatever. This is solved in regulated industries with "add-ons" (it's a huge stretch to call it that) like Veeva Vault that enforce formal document control workflows, but it doesn't help normal business users at all.

      • ch4s3 ・ 3 days ago
        ・ 5 more

        I'm strongly of the opinion that 1. is an anti-feature. It creates a ton of clutter.

        • bravetraveler ・ 3 days ago
          ・ 4 more

          Indeed. Instead of a dedicated set of channels for a topic... we have this breaking things up. This 'feature' absolutely kills focus.

          The old ways - curating channels - are the best ways.

          • ch4s3 ・ 3 days ago
            ・ 3 more

            It’s made worse by the fact that the search is strictly awful.

            • bravetraveler ・ 3 days ago
              ・ 2 more

              Which one? The one bound to ctrl+f or the one at the top of the window? /s

              Why are these even isolated? That's what parameters are for!

              • ch4s3 ・ 2 days ago

                It beggars belief

      • datadrivenangel ・ 3 days ago

        Teams spaces are just embedded sharepoint, which has many awkward edges.

  • scoot ・ 3 days ago

    > Slack have a rather good a11y experience

    But yet they hijack the standard Cmd-Z “insert link” shortcut for search, so you have to use their non-standard alternative. Not good.

    • gostsamo ・ 3 days ago

      IMHO, one or even a few suboptimal things do not invalidate all the rest.

MetaWhirledPeas ・ 3 days ago

Cypress has an excellent turn-key accessibility tool if you already use their product, and don't mind paying a lot for the accessibility feature. It uses your existing tests and makes evaluations based on that, with zero additional work.

amrocha ・ 3 days ago

I’ve set up automated accessibility tests like this at multiple companies I’ve worked at as well.

We didn’t have a dedicated accessibility team though, so I paired it with a shit list which the team worked through in less than a year.

quectophoton ・ 3 days ago

They still have that 3-4 second delay during login though. I open Slack directly from a browser, I login from a browser, yet I have to wait until they decide to ask me if I really want to continue with the browser I've been using for literally all the steps so far.

It's not much but it feels like when I'm on YouTube with a device that doesn't have adblock and a short ad plays before the video.

It's not related to accessibility, but it's still UX.

  • nicbou ・ 3 days ago

    > Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage.

    https://news.ycombinator.com/newsguidelines.html

    • quectophoton ・ 3 days ago

      My bad, I thought that rule was about the medium (i.e. the post itself). I didn't it also applied beyond that (e.g. commenting about Slack's UX on a post whose topic is a subset of Slack's UX).

      I'll keep it in mind, thanks.

      • SilasX ・ 3 days ago

        IMHO, your original interpretation is correct. The rule is about the UX of the article itself. In a story about Slack UX with respect to accessibility ... general issues with Slack UX seem topical enough.

  • egypturnash ・ 3 days ago

    Here is a way to get around this annoying behavior: Type “sla” in your address bar (or however many characters of “slack” it takes to get it to autocomplete with Slack URLs). Cursor down into saved URLs for actual Slack channels. Hit return. Skip all that bullshit.

    Discord does the same bullshit and this works there too.

    (This assumes you’re not logging out when closing the slack/discord tab. Sometimes it just doesn’t work with Slack and you have to do a full login.)

thdhhghgbhy ・ 2 days ago

A little OT but are people running playwright in docker on CI? Curious if it needs to be dockerised for consistent screenshots on local vs CI.

robertlagrant ・ 3 days ago

> Our automated Axe checks have reduced our reliance on manual testing and now compliment other essential forms of testing—like manual testing and usability studies. At the

Should be "complement".

traceroute66 ・ 3 days ago

Accessibility testing is important and great. However I wish Slack would put as much effort into general testing.

Some of their recent releases have left a lot to be desired.

sam0x17 ・ 3 days ago

ok cool but how about we get syntax highlighting for common programming langauges in code blocks?

rhzdgaf ・ 3 days ago

[flagged]

  • eckmLJE ・ 3 days ago

    Web accessibility describes specific requirements for people with disabilities to be able to use your website. If you don't implement these features, blind people, colorblind people, people who can't use a mouse, etc., won't be able to use your website. You can make a strategic choice not to support these users for reasons like ROI. But obviously there are plenty of situations where either we make the affirmative choice not to exclude people with disabilities, or we're required by law to accommodate them. In my personal opinion, we should always build on the web with the modern features that support assistive technologies, and that building inaccessible web experiences is synonymous with building poor quality web experiences. Many of the (mostly native) features that enable an experience accessible to people with disabilities improve the experience for all users.

    • msm_ ・ 3 days ago

      >we should always build on the web with the modern features that support assistive technologies

      Or build on the old web without modern features, that supports assistive technologies by default [1] :).

      [1] https://motherfuckingwebsite.com/

  • sunshowers ・ 3 days ago

    The fascinating thing about accessibility is that (unlike many other characteristics) all of us are going to be at least temporarily disabled at some point in our lives. I've really appreciated the presence of ramps whenever I've injured my foot, or just if I'm rolling a couple heavy suitcases.

  • TypingOutBugs ・ 3 days ago

    The EU has an Accessibility Act coming into play from June this year, so a subset of companies working on things like travel booking systems will have to test to make sure they meet those new requirements.

    https://www.epinova.se/en/blog/2024/understanding-the-europe...

  • LtWorf ・ 3 days ago

    Well it's mandatory to get government contracts. That's the only reason companies do it.

    They have no interest for disabled people.

    • sofixa ・ 3 days ago

      Not only government contracts, some very big orgs have accessibility requirements in all their vendor contracts too.

      • LtWorf ・ 2 days ago

        They do because of government requiring them to hire disabled people :)

  • ternnoburn ・ 3 days ago

    Your position is really "fuck disabled people", really?

    Your ever seen a blind person try to use the internet? You ever seen a person with essential tremors try to use their phone?

    Go and educate yourself a little.

    These standards are all built from the experiences of users who were unable to access things and they benefit way more than the 15% of people who have disabilities. Ever used closed captioning, automatic doors, or curb cuts? Those things all started as accessibility features.

LtWorf ・ 3 days ago

I'm the author of localslackirc, to bridge slack and IRC.

Probably most IRC clients are more accessible.

  • j16sdiz ・ 3 days ago

    If you consider the real-time text-only chat, maybe.

    As soon as you start thinking about the side features. they are not comparable.

    • LtWorf ・ 3 days ago

      for example?

      • msm_ ・ 2 days ago
        ・ 2 more

        You can't honestly suggest that IRC and Slack have feature parity. Slack has:

        * A support for sharing images and files

        * Rich text formatting

        * Can easily share code blocks

        * Rich and granular permission system, suitable for a large organisation

        * Webhook-based integrations

        * A rich ecosystem of existing corporate integrations (like calendar integrations)

        * Accessible web application, that provides access to all of those features

        * Mobile app (for both android and ios) that provides access to all of those features

        * Built-in OAuth/OIDC integration, that makes it easy to put it behind a company proxy.

        * User statuses, avatars, metadata (like real name or team name), timezone-awareness

        * Adding guests to channels, or bridging channels between servers

        * Voice calls

        * Search (!) with history, accessible for any device

        * Actually, you can write to someone who is not online right now, something IRC doesn't support without a bouncer.

        * Project management features (lists etc)

        * Well documented and rich API

        * Enterprise support

        And this is just out of the top of my head.

        • LtWorf ・ 2 days ago

          Remember that we are talking about accessibility, so the more images and animations and sound you have, the LESS accessible it is.

          Also we're not comparing IRC and slack, but slack used from an IRC client and slack used from an electron client.

          Perhaps you didn't completely understand what I was saying?

          > Mobile app (for both android and ios) that provides access to all of those features

          Nothing stops you from using all the clients you want? Again, I suspect you don't fully understand the thing you're hating on so hard.

          > Accessible web application, that provides access to all of those features

          Slow, and I'd like a disabled user to comment on that. Also not all features work on firefox.

          > Voice calls

          Doesn't work, but are you really claiming voice calls are more accessible than text? Have you asked a deaf person how they feel about voice calls?

          > Actually, you can write to someone who is not online right now, something IRC doesn't support without a bouncer.

          Works fine. Ok I realise you didn't read my comment and are just raging for no reason because you read "IRC" and got mad.

          > A support for sharing images and files

          Worksk fine with localslackirc. You can directly pipe the output of commands into channels

          > Rich and granular permission system, suitable for a large organisation > Webhook-based integrations > A rich ecosystem of existing corporate integrations (like calendar integrations) > Adding guests to channels, or bridging channels between servers > Well documented and rich API > Enterprise support

          ??? This has absolutely nothing to do with the client??

          > Rich text formatting

          Not supported

          > Can easily share code blocks

          Works fine, they're saved into text files

          > Built-in OAuth/OIDC integration, that makes it easy to put it behind a company proxy.

          Works fine with any authentication

          > User statuses, avatars, metadata (like real name or team name), timezone-awareness

          Avatar not supported, metadata is supported

          > Search (!) with history, accessible for any device

          Doesn't work, but it's of course accessible with other clients

          > Project management features (lists etc)

          If you do your project managing on slack, I'm glad I don't work with you :D

          I'm also glad I don't work with someone who just reads 1 word and gets triggered into an OT rant, to be honest.