Nooope, Baze doesn't keep anything like that! When he abandoned his faith, he also abandoned all trappings of it, including his staff, his lightbow, and his robes. However, his armor is red, like a Guardian's robes should be, so there's that.