The When Will I Be Famous? singer - who has become the third celebrity to leave the dance show - has been living with Poland Sydrome, which causes missing or undeveloped muscles in one side of the body and can affect the posture.
Goss, 54, told the Mail: "The costume team were relative strangers at the beginning but from the start they've always been very kind and discreet – it has really touched me. I have one pectoral muscle on the right side instead of two. The lower one is missing but the upper one is fine and strong."
"If I get a jacket made it will pull on the right side more. I have it tight on my right so it counterbalances my left side. Slim tailoring is good for me and my fashion is determined by my chest shape."
The singer – who rose to fame with 1980s boy band Bros – said the "very caring" wardrobe staff at Strictly helped make him outfits that made him feel comfortable and "elegant".
And Goss confessed when he was a pin-up in boyband Bros he learned to dress and pose to hid his condition.
Watch: Who is Strictly's Matt Goss?
He said: "Nobody will have ever noticed but I often used to pose covering that side of my chest."
He thanked the judges for their feedback, saying the experience had helped him improve his posture, overcoming the affects of Poland Syndrome.
He said: "My posture has improved since being on Strictly. I want to say to the judges: 'You've made me want to improve this and be proud of my posture'. I want to thank them."
Poland syndrome affects one in 20,000 babies, and twice as many males as females. It can often got undiagnosed in childhood.
And Goss hopes to start a charity to help young people with the condition know that it should not affect their confidence.
He said: "I’ve always been a confident man and I have a very strong upper body, but I know what it is like for kids with this condition... I am going to look at creating some kind of charity to build confidence, to show kids with the syndrome they can achieve everything in life.