Pain relief comes first, and supplements are add-ons that sit alongside it and never replace it, because they are not enough on their own and a cat with arthritis who is only on supplements or a special diet is very likely still in pain.
Supplements are classed as feed, so they are checked for safety and labelling but they don't have to prove they work, and they are not allowed to claim that they do, which means nobody has to show that a supplement relieves arthritis pain before it is sold, and the same rules apply whether the product is for a dog or a cat.
I don't want to completely dismiss them, as they may be a useful part of the picture once the pain is being treated. The guidelines say a systematic review found evidence to support omega-3 supplements and omega-3-enriched diets in cats with arthritis, and that joint diets may help too, so these are the ones I'd look at first.
Note that if your cat has kidney disease, a prescription kidney diet has good evidence that it helps cats live longer, so it may well matter more than any joint supplement.
Arthritis is slowly progressive, and every month spent waiting for a supplement to start working is another month of pain for a cat who could have been more comfortable, and with NSAIDs still available and frunevetmab now an option there is no reason to rely on supplements.
It's also genuinely hard to tell by eye whether something is working, because we all want it to, so a cat who seems brighter after starting a supplement isn't evidence, whoever is telling you. Keeping them lean helps their joints, although in a cat with kidney disease I'd take care over weight loss because losing muscle is a worry in its own right, so it is important to talk to your vet.
And if you are giving a supplement, please do tell your vet, so they know the whole picture.