on budget, if you are still paying on them.  If you have zero balance, and not making payments any longer, you can close them

It's already set up.  At any given time, the difference between your credit card balance and the credit card category balance is the debt that's been carried on the card.  You budget in your spending categories to cover purchases you'll make going forward, and you budget directly to the card's category to cover debt you incurred in the past. 

I set up the cards I'm paying off like my active cards, but I like the option to set a goal for paying it off in a certain time period. Then, YNAB tells me to put $300 toward it a month, so the orange bubble is a reminder that I still have to pay that portion each month. Can't wait till it's gone!

I have two that I'm working on paying down and what's worked for me is treating them almost like an installment loan- I have mine set up under tracking because I like to watch the balances go down, and I treat the payments like a regular monthly bill, ie scheduling a transaction in YNAB, just like a cell phone bill. This only works if you're TRULY not using them anymore, if you're using them at all they should go under Budget.