In February 2022, I was sitting on a beach in Hawaii during a hard-earned vacation with my wife. Now, that might be a common way to start a story for many of you, but for me, a vacation with my wife is actually a rare event due to my family dynamic with adult children with special needs (that may be a post for another day).
As an emergency physician entering my twilight years (I’m in my mid-50s), all the stress from the busy shifts with overwhelming patient volumes, the fluctuating sleep schedules, and the evenings and holidays away was definitely weighing heavy on my mind and body. I had just survived the worst of the COVID pandemic, although I felt like I had left a piece of me with it.
How much longer can I do this job? I had been asking myself that for years, but now I felt like I needed to make a choice: either retire early or find something else to do. I just couldn’t face many more years in full-time clinical practice. But I also didn’t feel like I was mentally ready to hang it all up and just go fishing. One piece of advice that I had heard repeatedly from early retirees was to be sure that you retire into something: know how you are going to spend your time, or you might find yourself unhappy and with nothing meaningful to fill the hours. Doctors are generally self-driven, highly motivated folks, and I am no exception. I felt like I still needed to contribute to something.
Long story short, I decided I would pursue a passion that I have developed over the years of investing and personal finance. I went back to school at my ripe old age (I am now finishing up a master’s degree in finance and the education requirement for the CFP designation) and completed the necessary licensing requirements to become a financial advisor, and I have joined an independent financial advisory group called Targeted Wealth Solutions (that I found through WCI). That is how I came to write this post.
I have learned a lot over the decades of real-world experience and the last year and a half of formal financial education. One of those things is how risky it is to retire—especially to retire early. Let’s talk about why.
Losing Your Salary
Medicine is a challenging but rewarding career. Most of us who went into it were excited by that challenge and by the opportunity to help others along the way. We were also thankful that it came with a reasonably safe expectation of high compensation in return for the many years of schooling, the accrued student debt, and the stressful lifestyle that accompanies that career. Our healthy paychecks certainly cushion much of the stress caused by life’s unexpected setbacks, such as the rising cost of living, health setbacks, home repairs, car repairs, education expenses for our kids, etc. For high income earners, these expenses are more like a bump in the road rather than the unsurpassable roadblock that others may face.
But once you retire, those regular paychecks aren’t going to be there anymore. You must now survive on the income that you generate from the assets that you have accumulated rather than the work that someone pays you to do. That can be scary. Really scary! I have always been pretty conservative with my spending, and I don’t live an extravagant lifestyle. My paycheck has made it easy for me to not lose sleep over finances. But maybe in retirement, that would be different. I haven’t worked for all these years to stress about money in my golden years. For me, I had to decide if I was ready to trade in the security of a steady stream of income that I could adjust as needed by simply working more shifts for a fixed retirement income where I had to be careful about not spending things down too quickly. That is why they say that a career in medicine comes with golden handcuffs. No matter how tired you are of the downsides of the job, it is hard to give it all up.
More information here:
Your Investments Are Your Lifeline in Retirement
Although there are exceptions, most doctors don’t get pensions and must turn the wealth that they have accumulated into income that will last the rest of their lives. If you were judicious with your earnings and you saved consistently, hopefully you will have a decent-sized nest egg when you finally decide it’s time to be done. But then, the accumulation phase is over, and the distribution phase begins.
There are a lot of questions to consider when making this transition. Did you save enough to last the rest of your life? How will you know? How much will you need to maintain the lifestyle you desire? What is the lifestyle you desire? How should you alter your investment approach during retirement so that you don’t spend down your savings too quickly? Will you have the flexibility in your spending habits to accommodate downturns in the market or other unforeseen expenses? These factors can drastically affect your chance of a successful financial outcome, which I would define as living how you want during retirement and then dying with some money in the bank that you can leave to your beneficiaries or charities of choice.
Before I became a financial planner myself, I had only rudimentary answers to these questions. Using the 4% rule and my current monthly budget, I made rough projections of my retirement needs. Now, I have much more sophisticated tools in my belt to answer these questions with more confidence.
The Risk of Bad Market Timing in Retirement
One of the biggest risks to your retirement nest egg is completely out of your control. You might simply be unlucky enough to want to retire at the wrong time in the market cycle. You could have invested exactly the same way as your best friend and got the same average return, but he retired when the markets were booming and your chosen retirement date happened to fall in a bear market. It is the decade surrounding this transition that is the riskiest to the size and longevity of your portfolio. This is referred to as retirement date risk.
History has shown that investing your money in the stock market generates the highest average return over the long term. But the laws of investing say that the highest returns come with higher risk. Risk can be defined by the degree of volatility—fluctuations from the average price. Stocks are much more volatile than investments with lower expected returns, such as bonds. Although you may enjoy an average return of around 10% if invested in the stock market, there are wide fluctuations from year to year; and some years, you can expect to lose money on your investments.
Hopefully by the time you approach retirement, if you have been investing regularly over all those years, you will have seen your portfolio value grow exponentially due to compounding interest. So, a 10% decline in the stock market (which is not that uncommon) can take a sizable chunk out of your portfolio (in dollars) if it happens close to retirement. Suddenly your account value is no longer what it was just a few months before. It might take years for it to recover, which means that you now will have to retire with less cash or you will have to push back your retirement date.
Conversely, if the market crash happens in the first few years immediately after you retire, your loss is compounded. Your assets depreciate, and you are forced to withdraw from them at exactly the wrong time to provide the income that you need to live on. In other words, you are forced to sell low on your investments. If you compare two scenarios where the total yearly returns of the market are identical over a 10-year period but come in reverse order, you can see how the timing of the negative returns is so important.
In my case, after years of a strong bull market, I was feeling pretty confident with my nest egg, and my target number was just on the horizon. Then, November 2021 happened, and the market crashed. I was invested in a stock-heavy, tech-centric portfolio that had done very well during the bull-market decade leading up to the crash. But those are the types of stocks that took the biggest hit during this down cycle. The NASDAQ, the closest index to my portfolio, gave back about one-third of its value in just a couple of months. Suddenly, my target number looked a long way away. Had that same drop happened five or 10 years ago, followed by a strong bull run, time would have made up for that loss, and I would have still been sitting pretty and feeling ready to make my beach vacation a habit.
This is a painfully personal example of the sequence of return risk. The only way to control for it is to decrease the volatility of your portfolio as you approach retirement and in the early years after you retire. That way, the impact of bad timing will at least be mitigated since the negative swing should be less extreme.
More information here:
Rising Inflation, Now?
The reason that we invest our money is so that it will grow in value over time. Yet, $100 dollars in 10 years will not buy nearly as much as $100 will buy today. That is because inflation, which averages around 2%-3% per year, is constantly eating away at the purchasing power of the dollar. This is referred to as inflation risk, and it means that, at the very least, your investment returns must beat the cumulative rate of inflation for your assets to grow in real (inflation-adjusted) dollars. We haven’t had to think too much about inflation in the last few decades, because it hasn’t been much of a problem since the early 1980s. But in the last two years, inflation has skyrocketed, reaching a high of 9% in June 2022. Like with bad market timing, if you happen to be unlucky enough to retire during a period of prolonged high inflation, the real value of your assets can take a major hit.
The Risk of Being Too Healthy
The vision that I think most of us have for retirement is travel, leisure, spending time with family, and exploring new hobbies. That can all be suddenly derailed by adverse health that keeps us from doing the things that we had hoped to do, and it is certainly something to account for in your financial planning scenarios. But have you ever considered the opposite? What if you are actually too healthy, meaning that you end up living longer than you had planned? In other words, what if you outlive your money? If you retire at the average age of 65, there is a 46% chance that you or your spouse will live beyond age 95. That’s 30 years of retirement! If you fully retire early, your investments may need to last for 40 or more years without any income to supplement them.
Typically in retirement, it is wise to change your investment allocation from growth-related assets to income-generating ones to start providing you a predictable paycheck and to avoid some of the high fluctuations in portfolio value. However, if you become too conservative in your investments and they do not keep growing in value and you live into your 90s or even early 100s, you risk running out of money or losing purchasing power due to inflation. This is referred to as longevity risk. To control for it, it is important to keep a decent percentage of your portfolio invested in stocks with long-term growth potential. Our firm uses an investment glide path that decreases portfolio risk during the decade surrounding retirement to address the sequence of return risk and retirement date risk, but it becomes more aggressive again afterward to protect against longevity risk.
More information here:
What If I’m Unhappy Retired? Can I Jump Back In?
Some of the risks of retirement are less tangible. As physicians, we are used to being needed and making a significant impact on others’ lives. After you retire, no one is going to pay you as much attention. You might feel unneeded and unfulfilled. I hear that it is quite common for people who have worked hard all their lives to find that not working is a major letdown. One of the risks of retiring too early is that it might not be easy to jump back into the workforce if you decide you aren’t enjoying all that free time. Medicine is always changing, and physicians can quickly lose skills if they are not constantly being honed. Depending on your chosen field, maybe you sold your practice and there really isn’t a good option for part-time work. It is great advice to try to leave your options open as you transition away from medicine or to find some other work or volunteer opportunity that will give you something on which to focus your attention.
The Bottom Line
I started this article on the beach in Hawaii pondering my future and calculating in my head the possibility of an early retirement. Let’s see: the market had just crashed, and my investments were down significantly. Inflation was climbing to a level we hadn’t seen in decades. I am fortunately very healthy and hope to live well into my 90s. And although I was ready to be done with clinical medicine, I didn’t think I would be happy without something else to fulfill me. My conclusion: it was a terrible time to retire. My solution: find something else that I enjoyed, go back to school, and launch a new career in my mid-50s. Not risky at all, right?
If you need extra help with planning for retirement or have
questions about the best way to save your money in tax-protected accounts, hire a WCI-vetted professional to help you figure it out.
What retirement risks worry you the most? How are you planning to mitigate those risks? Can you see yourself retiring early? Comment below!
[Editor’s Note: Dr. Bryan Jepson is a board-certified emergency medicine physician and a licensed financial advisor. He is finishing a master’s degree in Finance and Risk Management from the University of Colorado-Denver (graduation December 2023) and is concurrently preparing for the CFP licensing exam. He works for Targeted Wealth Solutions, an independent financial advisory and planning firm with a focus on healthcare professionals. He lives in Monument, Colorado, and he is married with three adult sons. He enjoys hiking, running, fishing, playing the guitar, and reading in his recently precious spare time. This article was submitted and approved according to our Guest Post Policy. We have no financial relationship.]