The new show - airing over six consecutive nights later this year - sees budding singers attempt to impress viewers and judges to win a cash prize, which increases the longer they decide to stay in the competition.
Cowell developed the show as an effective replacement for The X Factor in the ITV autumn schedules but has now decided to step back from being on screen in order to focus on his role as executive producer.
An ITV spokesperson said: "We are delighted that Gary has decided to join the Walk The Line family and we understand and support Simon's decision to focus on his role as producer and creator for this debut series.
"Gary will bring his musical expertise as one of the world's leading singer songwriters to the show, which promises to uncover some brilliant new talent and offer the unique chance to win a life changing prize."
Cowell added: "This is a situation where I have decided it is right for me to focus on my role as the creator and producer as we build up to the hugely exciting launch of the first series of our new show, Walk The Line.
"On that basis, I am thrilled that Gary is joining the panel. Gary's musical pedigree is second to none - and we have known each other a long time now - so I am delighted he is picking up the baton for me for the first series of Walk The Line. I know he will do a fantastic job."