The Pantex Savings Plan, administered through Voya Financial, is for many Pantex retirees the largest single financial asset they own — often more than their home equity, sometimes more than their pension. When you separate from service, you have three basic choices: leave it in the plan, roll it to an IRA, or take a distribution.

The "right" choice depends on factors most retirees don’t fully consider until after they’ve decided. Here’s the framework we use with clients in the Amarillo area.

Option 1: Leave the Money in the Pantex Savings Plan

What it gets you:

  • Continued access to the institutional fund pricing inside the plan
  • The ability to take penalty-free withdrawals starting in the year you turn 55, if you separate from Pantex in or after that year (the Rule of 55)
  • Stronger creditor protection in many states
  • No new account to open

What it costs you:

  • Limited investment menu (whatever Voya offers in the plan vs. essentially every public investment in an IRA)
  • Less flexibility on Roth conversions
  • Future required minimum distributions are calculated and administered through the plan, with less personalization
  • Beneficiary planning is constrained to plan rules

Best for: Clients retiring at 55–59½ who anticipate needing penalty-free withdrawals before 59½. Clients in states with weaker IRA creditor protection. Clients who genuinely don’t want a more complex setup.

Option 2: Roll the Balance to an IRA

What it gets you:

  • Full investment flexibility
  • Ability to do partial Roth conversions during low-income years
  • Easier consolidation with other IRA balances
  • Cleaner beneficiary planning

What it costs you:

  • You lose Rule of 55 access (IRAs require age 59½ for penalty-free withdrawals)
  • Some states offer less creditor protection for IRAs vs. 401(k)s
  • Subject to your IRA custodian’s fees, which may be higher or lower than the plan’s

Best for: Clients retiring at 60+. Clients who want active tax planning across multiple account types. Clients with significant assets who benefit from full investment flexibility.

Option 3: Take a Taxable Distribution

What it gets you:

  • Cash in hand

What it costs you:

  • Ordinary income tax on the entire distribution, often pushing you into a much higher bracket
  • A 10% early withdrawal penalty if you’re under 59½ and don’t qualify for an exception
  • Forever loss of the tax-deferred growth on those dollars

Best for: Almost no one, in almost no scenario. The exceptions are rare and worth running by an advisor before pulling the trigger.

The Rule of 55 Is the Part Most People Miss

If you separate from service from Pantex in or after the calendar year you turn 55, you can take distributions from the Pantex Savings Plan without the 10% early withdrawal penalty. That same money rolled into an IRA loses this benefit.

For someone retiring at 55, 56, 57, or 58, this matters a great deal. Rolling immediately means closing a door you may want to keep open. We often recommend a partial rollover — leaving enough in the plan to fund bridge years — rather than an all-or-nothing decision.

Outstanding Plan Loans: A Detail That Trips People Up

If you have an outstanding loan against your Pantex Savings Plan when you separate from service, the unpaid balance is generally treated as a distribution — meaning it becomes immediately taxable as ordinary income, and (if you’re under 59½) potentially subject to a 10% early withdrawal penalty.

You can typically avoid this by repaying the loan before separation or, in some cases, by depositing the equivalent amount into an IRA within the rollover window. Either way, an outstanding plan loan is something to address before retirement — not discover after the fact.

Roth Conversion Strategy

The years between retirement and Social Security claiming (or between retirement and age 73 for required minimum distributions) are often the lowest-income years of your life. Pantex retirees with $1M+ balances often have a 5–15 year window where partial Roth conversions can shift hundreds of thousands of dollars from "tax me later" to "tax me never."

But this only works if you have the funds in an IRA where conversions are mechanically possible. The conversion strategy and the rollover decision are two halves of the same plan — they have to be designed together.

Our Framework

Closing Thought

The rollover decision is reversible in some ways and not in others. You can move money from an IRA back to a 401(k) plan that accepts rollovers. But you can’t recapture a missed Rule of 55 window, you can’t un-trigger a deemed distribution from an unpaid plan loan, and you can’t make up Roth conversions you should have done in your 60s once you hit age 73 and RMDs begin.

For Pantex retirees in the Amarillo area, we help model all three options before the rollover paperwork is signed. The goal isn’t to roll for the sake of rolling — it’s to make the decision once, correctly.

“We can’t perfect the world, but we can show up for the people in it — and hold ourselves to a standard worthy of the trust they place in us.” — Ashby Dawson

About the Author

Ashby Dawson, CFP®, AAMS™ is a Partner and Wealth Manager at Kingsview Partners in Amarillo, Texas. She works with Pantex employees and retirees on pension elections, Voya rollovers, tax planning, and retirement income strategy. She has been recognized as a Forbes Best-in-State Wealth Advisor and Best-in-State Women Wealth Advisor in 2022, 2023, and 2026.

Considering a Voya rollover?

Schedule a confidential conversation: (806) 318-6820 | adawson@kingsview.com

Or book directly:

 

This article is for educational purposes only and does not constitute personalized financial, tax, or legal advice. Tax laws and plan rules change. Please consult a qualified fiduciary advisor regarding your specific situation before making any rollover or distribution decision.