The diffuser however had to be:
After getting recommendations on the FastLED group and SmartMatrix group, I ended up getting two things:
The Frosted Clear Plastic Cutting Mat was good for protection, but didn't offer too much diffusion. Still, it was better than nothing, and that's what I ended up using in the front of my shirt.
The Serenity Foam Underlayment for Laminate Flooring offered better diffusion, but no protection at all. However, I figured out that if I used that, plus the cutting mat on top, I got extra diffusion and protection.
If you wonder, the demos below can be had from:
Here are pictures of what it looks like with one and both layers:
The cutting mat only still provides some diffusion, just not as much:
Here is what it looks like with both layers and very small wires to hold the mat on top of the 3 32x64 RGBPanels attached together:
Here is what it looks like side by side, with both, and without any:
This works pretty well, thank to the people who provided suggestions and help