How To Preserve and Grow Your Wealth

Michael Flaxman
44 min readAug 16, 2018

--

Friends regularly ask for advice on how to manage their money, so I‘ve written up my notes to share with everyone. As an obvious disclaimer: I’m not a lawyer, accountant, or financial advisor. If you’re using this blog post to make decisions, you should consult one. My advice is free and there are no refunds.

This blog post‘s target audience is people in the US who are accredited investors (or on their way) and want to grow their wealth while managing their risk. If you have credit card debt or student loans, you’re probably better off focusing on paying that down before considering anything in this blog post.

Overview

Regardless of what you invest in, you should start by thinking hard about risk, diversification, and minimizing taxes/fees.

Risk Tolerance

You can often generate higher returns by taking on risk, but it’s not a free lunch. Most people want to chase risk when the market is up and are afraid to take on risk when it’s down.

As Warren Buffet famously said in his 2004 Letter to Shareholders (paraphrased):

Be greedy when others are fearful, and fearful when others are greedy.

Easier said than done. If the stock market fell 20% this year, would you buy more stock or sell some of your position (potentially at a big loss)? It’s a lot harder if you consider that you can’t know where the bottom is. The market could fall another 20% or more!

Most investors become more risk averse as their net worth increases. Once you have more money than you need, you focus on not losing it. Consistency is also very important. If a position falls 50%, it will have to increase 100% just to get back to where you started.

Whatever your risk tolerance, you should probably take less risk as you get closer to retirement. If risk-taking leads to losses, a young investor has more time to recover via investment, income, and cost reduction through lifestyle changes.

Diversification

Diversification is the only free lunch you get when it comes to investing. The basic idea is that you don’t want to put all of your eggs in one basket. What’s hard about diversification is that in order to be truly diversified you need to hold a large percentage of your portfolio in investments whose performance is not correlated with one another. In practice, this is limiting as many uncorrelated assets (cash, commodities, gold, etc) are unproductive and tend to have lower long-term returns.

In order to achieve diversification in the stock market you need to have a portfolio with many stocks in it; economists disagree on the exact number, but suggest at common guidelines is at least 40 companies. The odds you know more than the market for that many positions approach 0%, so one way to diversify is through the use of index funds. My favorite are Vanguard Funds, as they tend to have the lowest fees. VTI has an expense ratio of a minuscule 0.04% / year. Mutual funds can accomplish a similar goal, but they have some bad tax consequences. It may be easier to contribute a small amount from each paycheck to a mutual fund without paying transaction fees, so they’re fair game for tax-advantaged accounts.

An import disclaimer is that most rich people became wealthy by beating the market where they had expertise. If you’re confident your investments are going to outperform then it may be worth the extra risk to limit your diversification and focus on winners. Just be sure you’re aware of the risk you’re taking. As one popular expression goes:

Build wealth through concentration and preserve it through diversification.

Fees

Wall Street is very good at hiding fees. Like gas taxes, they’re often built-in and you won’t know you‘re paying them. Active managers charge management/performance fees, brokerages charge transaction fees, custodians charge storage fees, brokers charge closing fees, and the government collects taxes at each step.

Fees may seem small, but if you assume that your money will be invested for 40 years, then as little as 1%/year can add up to a huge chunk of your retirement going to someone else!

Here’s a simulation showing how much a 1% annual fee costs you on a $1 MM investment earning 6% annual returns:

Those fees add up to you working so someone else can retire! (source)

Of course the biggest fee is taxes, which is discussed in detail later in this post.

Note: For 95% of investors, what I’ve written above is all you need to know. For higher net worth individuals, DIY types, and optimizers, the rest is written for you.

Asset Allocation

Lazy Portfolios and Robo Advisors

There is a lot of research that suggests active managers can’t beat the market after taking into account their fees and the extra taxes paid from frequent trading. Why break your neck trying to beat the market when the experts can’t even do it? The concept of lazy portfolios has been around for a long time, and the latest version of that is robo-advisors like Wealthfront, Betterment, or Schwab’s Intelligent Portfolios.

If you have < $100k USD in the market, I’d recommend using a robo-advisor service. They’re very similar to what you should be doing on your own, they don’t make many of the mistakes you’re likely to, and their fees are very affordable (for smaller investors). All three will let you input your investment goals (without asking for your name, email, credit card number, etc) and give you recommendations. You could even copy their free advice and use their allocations in your own account. Note that if you do that you’ll still have to take care of rebalancing and tax-loss harvesting on your own (more on that later).

Here are example detailed allocations from Schwab and Wealthfront for two fictitious (and different) people. You can click on the assets and it will tell you the specific tickers that will be purchased on your behalf, with links to supporting rationale.

Sample Schwab Intelligent Portfolios recommendation
Sample Wealthfront recommendation

A similar (and older) option is Target Date Funds, but I prefer robo-advisors because they’re more tailored to your needs and have some tax advantages like tax-loss harvesting, Muni bonds (for residents of CA/NY), and even tax coordinated portfolios (in the case of Betterment).

Depending on your portfolio size and personality, you may want to use a Financial Advisor or Wealth Management firm. There’s a big adverse selection problem when it comes to paying someone else to manage your money; it’s a safe bet that whoever takes you on as a client doesn’t invest Bill Gates’ money. When you outsource to someone else you’re likely to pay a steep fee (1–2% of AUM/year and possibly even a percentage of performance) that will really add up over a lifetime.

Market Timing

Timing the market is really hard! If you can predict when the market is going to go up and down, you should buy and sell accordingly. You would be wise to start an investment fund or advisory service to profit off your amazing gift of being able to see the future.

Market timing comes with some large problems:

  1. When the market is up, the exuberance is contagious and you may feel that the bull market is just beginning. When the market is down and nobody is investing, will you be comfortable writing that check? For this reason, having an entry/exit strategy and sticking to it is often a wise choice. Otherwise, you may inadvertently sell low and then buy high.
  2. While you may be correct about the fundamentals, you can get killed by the timing. As economist John Maynard Keynes once famously said: “The market can stay irrational longer than you can stay solvent.”
  3. During bear markets, the best investments may be unproductive assets (more on that later).
  4. Each time you sell may trigger taxes (see section below). If your prediction about the market is incorrect (or just too early) this will add insult to injury.

Asset Classes

Depending on how much money you have to invest, there is a world of things you can invest in.

Determining your own allocation is very personal, and I’d suggest starting out by seeing what the robo-advisors recommend for your risk level and then tweak as you see fit. These typically don’t include investments like real estate, hedge funds, private equity, venture capital, commodities, lending, etc. As a general rule, the largest position in a growth-focused portfolio is typically US stock (for which some global market exposure is baked in considering that many US companies have extensive operations abroad), but after that there is fierce debate about percentages for the rest.

Stocks

The stock market is very liquid, has very low fees, and allows you to get exposure across many industries and geographies, often with low taxes. It has historically been a fantastic investment, and in the long run is effectively a bet on human productivity. Given the natural human desire to seek improvement, it would stand to reason that this will continue to be a good bet for many decades (if not centuries) to come.

The stock market can be very volatile for extended periods of time, but it has always produced strong returns over a long enough time horizon. You should never chase the upside in an investment unless you’re comfortable with the downside. This risk is why investors nearing retirement shift their portfolio toward more fixed income (bonds) and less equities (stocks). For instance, if you’re planning to retire next year and own 100% stocks and then a recession hits, you’ll probably have to postpone retirement and/or reduce your spending significantly.

(Source). Next time there is a bull or bear market and people say “this time it’s different”, please refer to this chart.

While index funds are a fantastic way to get broad exposure with low taxes and fees, market-capitalization weighting means that during a crash investors may find themselves over-exposed to the priciest stocks.

Bonds

In the event a company fails, bondholders are the first to be repaid. This makes bonds less risky and in theory their prices should not move in lockstep with stocks. However, in 2008 we saw both stocks and bonds fall together.

Interest rates are currently very low by historical standards, which has led to famous investor Howard Marks calling bonds “return free risk” (vs holding cash):

(Source). While interest rates have been rising lately, they’re still very low.

With bonds, the interest payments are taxed as income. This creates an incentive to otherwise lower your exposure or invest via a tax-advantaged account.

A general rule of thumb for stocks vs bonds is based on age:

% of Portfolio in Stocks = 120 - Your Age

So if you were 50 years old, then your portfolio should be 70% equities (120–50), with the remaining 30% in bonds.

Given the historically low bond yields we currently have and the fact that stocks traditionally provide a good hedge against inflation, I have a smaller percentage of my portfolio in bonds than this calculation suggests.

Note that there are some situations in which it makes more sense to hold individual bonds vs bond funds.

Cash (USD or Foreign)

Cash consistently loses purchasing power every year and is a horrible investment:

(Source). The USD is a terrible store of value.

The US Government sells Treasury Inflation Protected Securities (TIPS) that shield you from purchasing power loss. Whether you buy TIPS or some other related product, you are not betting that the value of cash will increase. Good reasons to buy TIPS include:

  1. You believe other asset types are going to crash. Be careful with this reasoning as you may miss out on substantial returns by sitting on the sidelines and waiting for a crash that never comes (or takes 5+ years). You might be better off deploying some of the cash in investments you think will perform well in a recession. For example, I bought COST stock in 2017.
  2. You have come into a large amount of money at one time (e.g. an inheritance or the sale of a large investment) and you don’t want to try to time the market. You might do better deploying 10–30% of it per month in case the market collapses the day after you make your investments. This form of entering/exiting a position is known as Dollar Cost Averaging and is generally a good investment strategy.

There are lots of complex financial products to protect investors from downside for a fee, but a much simpler and lower fee approach is to hold some percentage of your portfolio in cash. However, in the long run that portfolio will still underperform a fully-utilized portfolio. It only makes sense to hoard cash if you plan to deploy those cash reserves aggressively during a downturn. There’s no reason to be punished for holding cash in good times unless you’re going to use it in bad times.

Real Estate

Buying real estate can be a fantastic investment, but it doesn’t always make sense. Given that capital can move relatively freely across the globe, what are the odds that the best investment in the world is within a short drive of where you live?

As we learned in 2008, housing prices can go down. Real estate ownership has significant transaction costs, property taxes, and carrying costs. If it’s your primary residence, it becomes harder to move if you get a better a job offer or need more/less home (perhaps your family grows or children move out).

In general, it makes sense to buy if your needs are constant for a long period of time (e.g. if your family is not growing/shrinking and you won’t want to move to a different area). The exact amount of time where home ownership makes more sense depends on macro conditions and the economics of your regional housing market. You can use a free online rent vs buy calculator like this one or this one.

If your house would be easy to rent out for more than the mortgage (plus taxes and maintenance), the math for buying makes a lot more sense since this makes moving less difficult. This can be great because it generates cash flow (taxed as income) that can be used to fund your lifestyle. Real estate is one of the few investments that most people can get leverage for (through mortgages), which can amplify your results for better or worse. Buying because you think the price will go up can be a dangerous game of musical chairs. Even if your rental income is currently higher than the mortgage, there’s no guarantee the price will hold.

If you do buy a house, it’s probably a bad idea to have your entire net worth in one investment. A general target is<40% of your portfolio in real estate, preferably spread out across multiple properties, potentially even in different geographies.

REITs

One way to get exposure to real estate is through a Real Estate Investment Trust (REIT). REITs are a legal structure that pool many investors money to purchase real estate. These investments generate passive income and pay out at least 90% of it as taxable income.

The benefits of investing in REITs vs owning property directly:

  • Diversification
  • Don’t have to be a landlord, make improvements, deal with tenants or admin work
  • Accounting is simple and done for you

The negatives:

See this article for a decent overview.

Commodities

Commodities are physical goods that have intrinsic value because they can be consumed. Examples include metals (copper, silver, palladium, platinum, etc), energy (gasoline, heating oil, crude, etc), and food (cattle, hogs, wheat, soybeans, etc). Lots of financial instruments like futures contracts let you bet on the future prices of commodities without ever having to take possession of the physical goods.

Like monetary goods, commodities are an unproductive asset (i.e. they don’t generate interest or dividends). The main reason to hold them would be that you believe the market is underestimating future demand and/or overestimating future supply. You can still get some exposure to underlying commodities via investments in productive stocks in that industry. For example, copper miners will do well if the price of copper increases.

Some advisors recommend a tiny allocation, but I’m skeptical and think they do this to appear more sophisticated/useful. In most cases, if the price of a commodity increases, commodity producers will create/mine more of that commodity to profit from that price appreciation. Then, the price naturally falls back to its original level.

Gold

Gold is also an unproductive asset, since it doesn’t earn any income.

Here is what legendary investor Warren Buffet had to say about gold:

You could take all the gold that’s ever been mined, and it would fill a cube 68 feet in each direction. For what that’s worth at current gold prices, you could buy all — not some — of the farmland in the U.S. Plus, you could buy 16 Exxon Mobils, plus have $1 trillion of walking-around money. Or you could have a big cube of metal. Which would you take? Which is going to produce more value?

Buying gold is effectively an “expression of fear”. It’s often used to hedge against rampant inflation or a zombie apocalypse. Since these outcomes may lead to the breakdown of society, you want to hold the physical gold in a vault or bank safe deposit box that you have access to and not an ETF like GLD. In the event you actually need your gold, a claim on gold in a vault across the world may be of little value. It’s even worse if you’re “holding” unallocated gold.

One unfortunate thing is that physical precious metals (coins, bars, ingots, etc) like gold are taxed at a much higher maximum 28% long-term rate as of 2018, not the lower long-term capital gains rate. That doesn’t seem fair at all, but I don’t make the rules. Gold is often passed down across generations, so you may be able to get around this if you die with less assets than the estate tax threshold (see section on the Estate Tax) for more details.

Gold is very hard to create (hence the term “hard money”), and this is why gold has a high stock-to-flow ratio. If demand for gold increases, it is difficult for gold producers to increase supply to meet this demand. This is not nearly as true with silver for example.

Holding 1–2% of your portfolio in gold might make for something of an insurance policy against a global catastrophe. But before you do that you should probably have an escape plan from where you live to a place that’s off the grid, far from any major city, and well stocked (water, food, medical provisions, guns, ammo, etc).

Gold has sometimes done well during periods of turmoil, but not always. It’s probably a better hedge against crisis than inflation (where you can buy a productive asset like stocks which are a natural hedge against mild inflation):

(Source). It’s hard to figure out what makes gold prices move.

Bitcoin

Bitcoin (and cryptocurrencies generally) is a new asset class that has existed for only a decade. Bitcoin offers returns that may be uncorrelated to the market, but like gold it is an unproductive asset. It is effectively a digital gold, but more scarce, portable, divisible, and harder to counterfeit.

For a better understanding of bitcoin I recommend you read Vijay Boyapati’s excellent blog post series The Bullish Case for Bitcoin which lays out why it’s such an asymmetric bet. Saifedean Ammous’ book The Bitcoin Standard is an excellent read but is much much longer.

Holding 2–5% of your portfolio in bitcoin is probably a wise idea in case it eclipses gold (or one day even becomes the world’s reserve currency). There will only ever be 21MM bitcoin, so if you have the ability to own a few you might find yourself in a very fortunate position in the future.

For more information about how to store your bitcoin securely, please see this blog post I wrote.

Startup Investing

So you want to be an angel? These are the drivers of startup opportunities:

  1. New technology. The internet/mobile wave has passed. Now, there’s bitcoin, 3d printing, VR, etc. It feels much more niche than what’s come in the past (society moving to a digital world), though I do love bitcoin.
  2. Increase or decrease in government interference, which is generally making something legal (marijuana) or illegal (green energy benefiting from subsidies). Can border on shameful cronyism, and is relatively niche. In the case of subsidies, even government money isn’t enough to guarantee success (see Solyndra).
  3. Cultural change. These are hard to predict and generally smaller as culture shifts slowly. Uber/AirBnB are good examples here, though this is not usually strong enough; you need a combination with one of the above to create large opportunities.

There are significant downsides:

  1. Scale. Even professional investors expect the majority of their investments to lose money. In order to have any reasonable chance at a positive return, you need to make many investments. You probably lack the time, deal flow, and potentially capital to do that effectively.
  2. Deal flow. What are the odds that the companies you cross paths with are going to be the next Google, Facebook, or Uber? You might have a chance if you run an accelerator like Y Combinator, are a famous Silicon Valley investor like SV Angel, or a big VC fund like Andreesen Horowitz. Statistically speaking, your college roommate’s cousin’s friend is not the next Mark Zuckerberg.
  3. Complex deal terms. Startup investing is more complicated than public markets because you’re not just backing a company, you’re also facing unique terms for each investment. While many early-stage companies use standardized language in SAFEs or Convertible Notes, there’s still lots of room for custom terms and side-letters. Seed investors often have to give up their rights to larger investors in future rounds.
  4. Liquidity. The lifecycle of a startup investment can often span 10 years, so you want to be sure you won’t need that money in the future (to buy a house, send a child to college, pay for a medical bill, etc). During that time you’ll have limited information rights, and your hot startup can turn into the next Theranos.

If you want exposure to this asset class, your best bet is probably to be an LP in a VC fund. You must be an accredited investor, the minimums are high, and the most coveted funds are not taking on new investors. You’ll also probably pay a 2% management fee and 20% of profit across the fund (aka “fund carry”). You should be very skeptical; since the late 90s even top performing VC funds haven’t beat the market and average funds lose their investors’ money:

(source) Not surprisingly, every fund you talk to will tell you that they’re in the top quartile.

Platforms like AngelList help investors form startup “syndicates” to invest in part of a company’s round. The upside is that you can make tiny investments (as small as $1k) in many companies and it’s as easy as clicking a button. The downside is that you pay the syndicate lead (who often “leads” by putting in a negligible percentage of the syndicate’s money) ~20% of the upside on your investment, structured as deal carry. This means that if 9 out of the 10 investments with this syndicate lead fail and one returns 10x, you’ll only break even before fees, but will have to pay a 20% fee on that one winning investment to the syndicate lead for the privilege of (now) losing your money.

If you want to invest some of your money in startups for fun, then by all means go for it; just be sure to recognize that what you’re doing is effectively gambling. Some use it as a means to give back and mentor entrepreneurs, some see it as a way to stay on top of the latest trends, and some just want to be able to brag at cocktail parties about the hot startup they just invested in.

Buyer beware.

Hedge Funds & Private Equity (PE)

Hedge funds were originally a mechanism to hedge risk by going long in one position/vertical and short in another, often while taking on lots of leverage. They charge high fees, typically ~2% annual management fee + 20% of performance. Hedge funds now exist to service every niche and investment strategy across stocks, bonds, private companies and even cryptocurrencies.

Private Equity’s main historical use case was to invest in private companies on a deal-by-deal basis. Since PE and hedge funds share many similarities (a partnership for accredited investors with high fees/investment minimums), I’ve included them together for reference.

For most people, the main issue with these vehicles is that their minimums are prohibitively high. Many funds have minimum investment sizes of $500k or much more. If you do have that kind of money, it can be hard to pick which funds you think will outperform.

Taxes

“Nothing is certain except for death and taxes.” -Benjamin Franklin (source)

Minimizing your tax liabilities can lead to a dramatic increase in your bottom line. Unlike most trades which involve risk, a change in your business or trading strategy to reduce your tax burden is nearly a sure thing, provided:

  1. You’ve received proper tax advice and followed it correctly
  2. You’re not living in a Banana Republic that will change the rules after the fact, which famously happened in California in 2013.

Tax-Advantaged Accounts

Tax-advantaged accounts (sometimes called “qualified plans”) provide two benefits:

  1. Instead of paying taxes on dividends (for investments held) or capital gains (when trading) each year, you only pay taxes when you withdraw from your retirement account (or in the case of a ROTH IRA never as you already paid taxes on those funds when you contributed). This compounding can really add up, as you’re able to reinvest funds that otherwise would’ve gone to the government over the years on account of realized gains, dividend payments, rental income, partnership profits, etc.
  2. There are often tax benefits for participation. If you’re an employee, you can contribute up to $18,500/year of your salary to your 401k as of 2018 and your employer can elect to make matching contributions of up to $36.5k/year as of 2018 (though few employers do), for a combined total 401k contribution limit of $55k. If you’re self employed, you can contribute up to $55k/year as of 2018 to your SEP IRA. However you do it, this is a potentially huge reduction in taxable income.

The biggest negatives are:

  1. You don’t have access to the funds until you are ~60 years old, unless you want to face steep penalties. Some hardship exemptions exist, but you shouldn’t contribute unless you’re expecting to lock the money up for your future.
  2. Under current rules, you must start taking minimum distributions at age 70.5 as of 2018. If you’re retired then and want access to those funds, this may be no problem.
  3. You’re implicitly trusting the government to not steal this money in the coming decades. In the event of government bail-in, the most likely scenario would be based on account size, so this is probably a better argument for tax-advantaged accounts to be a smaller % of your portfolio. It also may make sense to only contribute up to the limits (above) for taxable income reduction. That way, if the government later robs you of some of your retirement funds you are likely to have locked in the earlier benefit of lower taxes when you made the contribution.

If you have a self-directed account (like Pensco or Rocket Dollar), you can invest in all kinds of asset classes (hedge funds, real estate, etc) and still take advantage of these tax benefits. A good tactic is to invest in assets that generate a lot of income, capital gains (especially short-term capital gains), and dividend payments, via your tax-advantaged account; you’ll get the benefit of diversifying into those investments without the tax consequences. Be careful that there are gotchas in the accounting for more complicated investment vehicles; you may be subject to UBIT in a REIT or an MLP!

Tax Loss Harvesting

If you have a capital loss (i.e. you sell an investment for less than your cost basis), you can usually apply it against another capital gain to reduce your tax bill. If capital losses exceed capital gains in a given year, you can apply up to $3,000 as of 2018 of this excess loss to offset against your income, which is even better as ordinary income is taxed at a much higher rate than capital gains. Any capital losses you don’t use in one year can be carried forward for future years, though it may take you a long time to use them all up.

If you have a position that is worth less than when you bought it, it is generally considered a “wash sale” if you sell it and then buy it back within 30 days, so don’t do this. However, if you bought some Coca Cola yesterday and it is down 1%, you could sell it today and instead buy some Pepsi stock, betting that the two generally move together (the Pepsi stock might also be down the same amount since the previous day). You’d realize a loss on your Coca Cola and have no taxable event for holding Pepsi until you sell.

Similarly, there are index funds that may be considered substitutes because they have very similar exposure, although their exact composition and size means they are not identical:

(source). I’ve created a google spreadsheet with similar data as well.

It’s important to note that there is some debate about how the IRS might treat this in the future.

Tax-loss harvesting is a real pain, and robo-advisor services like Betterment, Wealthfront, and Schwab’s Intelligent Advisor will do this automatically for you. In the simplest scenario, if you deploy a lot of cash and then see the price go down substantially, you may want to harvest some of those losses right away.

TLH sounds boring, and if the only benefit were delaying an inevitable payment to Uncle Sam I’d say it’s good from a time-value of money perspective but not amazing. However, it has some huge benefits that are overlooked:

  • If you live in state with a high state income tax (hello CA and NY!), your capital gains are taxed as income at the state level. This means that you could harvest losses today that would count against other state taxes now ($3k against earned income and an unlimited amount against other capital gains), and years later move to a retirement-friendly state with no state income tax (see State Residency section) when you retire. If you then sold that assets with a larger gain (remember it now has a lower basis from tax-loss harvesting), you’d only pay federal capital gains and never pay any state capital gains on that investment.
  • If you hold that investment until you die and are under the estate tax limit (see Estate Tax section), then your heirs will receive that asset with a stepped-up cost basis when you die (meaning taxes are never paid on the gain from that investment, including from the time of TLH). In this case, you get the tax benefit today while giving up nothing in the future! Of course for this strategy to work you have to one day die without selling off that position.
  • It’s risk-free. Unlike a ROTH IRA where you pay taxes today in hopes that the government will honor the agreement for you to not pay taxes in the future, with TLH you get the benefit today expecting to pay in the future. There’s always a chance there will be new tax programs in the future you can use to later defer/minimize those gains (like qualified opportunity zones), and TLH provides that optionality.

Progressive Taxation

The US has a system of progressive taxation, which means that generally the more you make the more you pay. This is true for both income and capital gains.

One strange result of this, is that it’s usually better to make $X each year for 2 years vs 2 * $X in one year. The reason is that you may end up paying more taxes overall. For example, if you only want to work every other year, you would be better off to start/stop your work in June instead of January to smooth your earnings across calendar years. Lottery winners famously tend to get this one wrong; by requesting a lump-sum payment (vs payments over many years), their effective tax bill is much higher.

Obamacare/ACA (Net Investment Income) Tax

If you make over $200k for single people ($250k for married couples) as of 2018, then 3.8% of whatever you make beyond that goes to paying for Obamacare. There are some complications for how this NII number is calculated, see this complicated guide from the IRS for the official details.

Capital Gains

Capital Gains are also taxed at progressive rates. If you realize less than ~$426k (plus income) if you are single (and ~$479k for married couples), then as of 2018 your capital gains rate is 15%. Above that threshold, capital gains are taxed at 20%. So, if you have a long-term stable asset with > $426k in unrealized gains (or less if you have income) and you want to sell it, you may be better off to split the sale across multiple years as a form of dollar-cost averaging. Don’t forget about the 3.8% Obamacare taxes on those gains as well!

Municipal Bonds

Muni Bonds offer some tax advantages, and if you live in high tax states like NY or CA you can even get a local one that invests in government bond offerings in your state (of course limiting to a single state will reduce your diversification). These muni bonds are not 100% tax free.

1031 Like Kind Exchange

If you own real estate as an investment property and exchange it for another home that is “like kind”, you may be able to defer any taxes on that transaction and keep your old cost basis in the new property. There are a lot of restrictions and hoops to jump through, but it’s definitely worth considering for an investment property. The open secret in real estate is that if you roll your investment property into another you may not pay any capital gains taxes for decades.

The IRS doesn’t like you to do 1031 exchange on your own and forces you to use an exchange facilitator company that charges money for their services (generally $800-$1,000 per transaction). If the seller of an asset touches any proceeds from their capital gain it disqualifies the exchange. They also must invest the whole proceeds (not just the gain) in their new project.

Primary Residence Capital Gains Exclusion

If you sell your primary residence (generally defined as having lived in it for 2 of the previous 5 years), then the first $250k of capital gains if you’re single ($500k for married couples) as of 2018 is currently tax-free. This benefit is less useful for extremely expensive houses; if you buy a $10MM house and have a $10MM gain over 20 years, then (assuming the law doesn’t change) sheltering only 2.5%-5% of the gain isn’t that great.

Rebalance through non-reinvestment of dividends

Most stocks (ETFs) produce dividends which investors need to pay taxes on. The exact amount of dividend payments depends on the company, sector, and performance that year. For example, the S&P 500 typically pays out ~2% in dividends each year for the past two decades:

(source) These dividend payments may be especially important for retirees.

Most dividend payments are “qualified dividends”, meaning they are taxed at the same rates as long-term capital gains, but some dividend payments are “unqualified dividends” and subject to income tax rates which are higher.

By default, most brokerages will automatically reinvest the dividends in your original position (with no transaction fee). Unfortunately, the government still sees this as a taxable transaction and is going to want a piece of your long-term investment. Since you can’t avoid these taxes, you may want to instruct your brokerage to set your dividends to not automatically reinvest. The downside of this approach is that you’ll have to actively choose how to deploy the capital (perhaps to rebalance and buy more of your underperforming stocks). This strategy also works for those who use the dividends as their main income source to fund their lifestyle (e.g. retirees).

In a tax-advantaged account, you’ll want to do the opposite and reinvest the dividends as it keeps your capital deployed automatically with no transaction fees. If you want to rebalance later there’s no taxable event to sell one position and buy another.

Foreign Tax Credit

At the end of the year, your brokerage’s financial statement will list “Foreign Tax Paid” by your investments, which you can credit against your US taxes. Be sure to pass this along to your accountant who will probably fill out Form 1116.

Health Savings Account

HSA healthcare plans have nice tax benefits, but they are not right for everyone. The main benefit is that you can contribute up to $3,450 if you’re single ($6,900 for families) as of 2018 and this reduces your taxable income. These funds can only be spent on “qualified medical expenses” (full list here; it includes most things you would think of as medical care), but the funds never expire and there are never any taxes. Even better, you can invest unspent funds in the market with no tax consequences. Some take the extreme approach and use HSAs as retirement accounts; they avoid reimbursing themselves for qualified medical bills each year. Since there’s no deadline to reimburse yourself, you can invest the money (tax free) for decades before finally reimbursing themselves for those qualified medical expenses much later in life. I don’t recommend this approach as I’d be worried that the law could change in the future, or the government might change their stance on this.

The main downside of HSA accounts is that they’re only eligible for high-deductible health plans, which I personally prefer but are not a fit for everyone. Also, many companies don’t offer HSA-compatible plans, or if they do they also offer much nicer plans that are heavily company-subsidized meaning that the after-tax savings can be negative to the employee.

Regardless of what type of healthcare plan you choose, if you’re self-employed your insurance premiums are likely tax deductible.

Income vs Capital Gains Tax Rates

The difference between the top marginal federal income tax rate (37% as of 2018) and the top federal capital gains (20%, also subject to an additional 3.8% Obamacare tax) has major implications for growing your wealth. Let’s say you you can either earn an extra $X by working Y hours, or you can spend the same extra Y hours investing your money better and generate an extra return of $X. Which would be a better use of your time? Since you pay much higher taxes earned income vs capital gains, the latter is better (all else being equal).

This is of course only true if you have enough assets or are good enough at investing that you can make similar money investing as you could working.

Similarly, you can think of management/performance fees as forgoing $X of investment gains (and thus reduce your capital gains) to save Y hours of your time; but you’d actually be better off if you could save those same Y hours by paying someone the same $X to help you with your business (as a business deductible expense). Again, all things being equal you should prefer the business expense, since the deduction is counted against ordinary income taxed at much higher rates than capital gains.

The easiest improvement for some is to reduce post-tax spending, instead of working for extra ordinary income:

Yup, I quoted myself.

Marriage Tax Penalty (and Bonus)

Due to the nature of progressive taxation, dual-income couples will most often have a higher tax bill if they get married. It’s somewhat bizarre that a culture that highly values marriage (generally) financially penalizes it.

The specifics are beyond the scope of this article, but the Tax Policy Center has a good marriage tax calculator.

One important benefit is that the when a spouse dies, the surviving spouse can receive all of their assets tax free (even if the amount is greater than the estate tax exemption). So, if you have greater than $11.2MM USD and are in a long-term committed relationship, there is extra incentive to marry your partner. Consider it like a free life-insurance plan, courtesy of Uncle Sam!

529 Plans (College Savings Plan)

529 plans allow you to contribute post-tax funds toward a child’s college education, and any investment gains are never federally taxed in the future as long as they’re spent on qualified education expenses. As of December 2017 you can use these funds to pay for K12 private schooling as well, although if your state has an income tax then using the funds for K12 education may trigger a penalty.

The rules are a bit complicated and it pretty much only makes sense if you’re wealthy enough to know you won’t need any help paying for schooling. This is ironic given that it’s “designed” to help people afford education.

Pros:

  • Tax-free investment returns can yield large savings.
  • While the beneficiary is typically your child/relative, unlike an irrevocable trust you remain in control of the account and can even change the beneficiary.
  • If your state has an income tax, you may get some benefit for your contribution, though restrictions apply (some details here and here)

Cons:

  • 529 plans usually have higher admin costs, meaning they make more sense for longer time-horizons so the savings from compounding are larger. They also typically have fewer fund options to choose from.
  • 10% penalty (on gains only) if the funds are not used for qualified expenses, plus gains will be taxed as income; this could happen because your child doesn’t go to college, or you save too much money. There are restrictions on qualified expenses too (e.g. you cannot spend any extra funds on a car for your child). You can change the beneficiary to another child (or even yourself), and there is an exception if the child dies or becomes severely disabled.
  • Having a 529 plan may lead to a student receiving less financial aid, whereas an IRA won’t. However, parents of students who expect to receive financial aid may be better off without a 529 plan anyway. You can get a loan for college, but you can’t get a loan for retirement.
  • Gift limits for estate taxes still apply, but growth in principal is always tax free. You can also make a lump-sum gift of 5-years to start putting funds to work immediately.
  • As with all tax -advantaged accounts, there’s always the risk the government will change the rules after the fact. The risk is lower here since the time horizon is shorter vs a retirement account.
  • There are limits to how much you can contribute, and there are a host of minor rules/restrictions you’ll want to familiarize yourself with.

It’s probably best to think of a 529 plan as a supplement to a an irrevocable trust. The main benefits over an irrevocable trust are that the invested funds compound in a tax-advantaged account, and the account holder maintains control of how the funds are invested and for whose benefit after the gift.

Charitable Giving

If you have an asset that has appreciated substantially and you want to donate to charity, the charity is far better off if you gift them the asset directly vs if you sell the asset and gift them the proceeds. Recognized charities can receive the entire gift in-kind and sell it with no tax consequences. However, if you sell the asset first, you’ll have to pay taxes on the sale and will have less money left over to give to the charity. Another consideration is to use the donation to smooth income and reduce your tax bill in years when your tax rate is highest.

Private foundations involve a ton of paperwork and ongoing administrative costs, so donor-advised funds have become popular for charitable giving as they require very little effort on the donor’s part. Fidelity, Vanguard, and Schwab all offer donor-advised fund products with low fees.

Taxes and Residency

Sometimes, the best way to minimize your taxes is to move. For many people, moving may not be an option. It often makes the most sense for retirees who are living on a fixed income, are no longer tied to their former job location, and are realizing lots of capital gains. Hello Florida retirement homes!

Places with low income/estate taxes often have higher property/sales taxes. These offsetting taxes are still generally much lower, and are also much less progressive. Bill Gates might be over a thousand times richer than you, but his house doesn’t cost nearly a thousand times more than yours!

State Residency

Each state has different taxes, and you’ll be subject to the taxes of where you choose to live. States tax capital gains as income according to their state income tax rates.

(Source) Some cities (like NYC) even have their own income taxes as well.

The most popular cities in states with no income tax are Austin, Seattle, Miami and Las Vegas. Other popular locations include Houston, Dallas, Jacksonville, Reno, and Lake Tahoe’s North Shore.

One common example is for the founder of a company to move to another state before selling stock in their company. The stock may be for a NY company, vested while they were a NY resident, and held for many years while they lived in NY, but if they legitimately now live in an income-tax free state they may be eligible to pay $0 state income tax on that capital gain.

Residency to an auditor is much more than just where you reside, and there are tons of things you must do in order to be eligible. There are a lot of complex rules about this so you should definitely consult a tax professional. A lot of it is about severing ties with your previous state (real estate, vehicles, voting, bank accounts, safe deposit boxes, private club memberships, doctors, lawyers, accountants, fishing licenses, etc), not just establishing ties to your new home. At a minimum, you will want to flee your state in the tax year preceding any transaction, and be well over the minimum 183 day requirement.

Country Residency

The US is one of the only countries that taxes your income worldwide, so living abroad will not reduce your taxes. The only exception is that if you live abroad, then the first ~$104k of your foreign income is tax-free as of 2018. Even if you have other citizenship and give up your US citizenship, you still have to pay a large exit tax, fill out a lot of paperwork, and pay a $2,500 filing fee.

If US government spending continues to grow without restraint, the country could face a serious decline in the long run. Add in unfunded liabilities, switch to accrual based accounting, and factor in a future of increased inequality and the US seems like it could be following in Greece’s path (or worse) in the decades to come:

GDP has exploded during this time period, so the true picture is much worse (source)

Historically, it’s very hard to flee a country during times of political unrest or war. It might make good sense to get foreign citizenship a decade in advance (while it’s relatively easy) to be safe. Popular choices include New Zealand, St Kitts & Nevis, and Malta, but YMMV.

Territory Residency

Puerto Rico is unusual in that bona-fide residents can pay $0 in federal income tax by living in the territory. Territory taxes can be as low as just ~4%, which is an absolutely massive discount and much greater savings vs moving to a state without state income taxes.

There are a number of negatives to this approach that I’ve heard. Take this with a grain of salt as I’ve never been to the island:

  • You must commit to being a resident of the territory for 10 years. In theory, if you move back to the mainland you might have to pay back the tax savings, though I’m not sure how often this is enforced.
  • You must mark your assets to market and pay mainland taxes before you are eligible for Puerto Rican tax rates. This means you can’t show up with a massive unrealized gain and then sell a large position.
  • Puerto Rico is constantly asking the mainland for a bailout, and one day they may get one. If they do, it will likely come with the requirement that they close the loophole for avoiding mainland taxes. If you’re going to move across the world for low taxes, be sure you’re actually going to get them!
  • Puerto Rico is far from the mainland and you may become isolated from many of your former friends.
  • Puerto Rico requires you to apply for bona-fide residency, which takes some time. You also have to invest in local businesses that meet certain criteria.
  • The island is not an economic hub. College graduation, literacy, english-speaking rates, and internet penetration is much lower vs the mainland. Alcoholism and unemployment rates are higher.
  • Puerto Ricans cannot vote in US elections.

Estate Tax (Death Tax)

Trump’s Tax reform increased the Estate Tax threshold in 2018 to $11.2MM USD (2x that for married couples), meaning it applies to an even smaller percentage of Americans. A person with assets greater than this may be tempted to give money away before they die, but can only gift $15k/year (2x that for married couples) per recipient as of 2018 before it becomes taxable.

At death, the cost-basis of an asset is “stepped up” to the current value. So if your rich uncle has some stock they bought for $1 a long time ago and it’s worth $100 when he dies, the cost-basis when you receive it is now $100. Perhaps you can avoid death OR taxes, just not both!

Also, some states have an estate or inheritance tax:

Try not to die or receive an inheritance while living in one of these states! (source)

Random Tidbits

The following sections outline other considerations that didn’t fit neatly into the previous categories.

Which Brokerage to Use

The ideal brokerage is an institution that is too big to fail (implicit government guarantee), has aligned incentives, reputable auditors, and a large insurance policy. The best choice is probably Vanguard; as a mutual company they have long-term interests that are aligned with their customers. However, Fidelity, Schwab, TD Ameritrade and Merrill Lynch all have > $500 BN AUM (some much more), so they’re probably safe as well.

Ultra low-cost trading options like Robinhood (warning: relatively tiny AUM as of publication) have helped bring down transaction costs across the industry. For long term investors who trade infrequently just to rebalance their portfolio, these costs should be minor.

For active traders who transact often, another factor may be commissions charged per trade. Interactive Brokers (which has over $50 billion AUM) is popular among professional traders.

Historically, many brokerages (Fidelity, Schwab, Merrill, TD Ameritrade) offered cash incentives and free trades if you brought significant new money into your account. These offers are a nice bonus if you already plan to deposit funds into your account (perhaps from the sale of a large position that isn’t in your brokerage account). A common offer across various brokerages is a $2,000+ bonus for $1MM of new money deposited (and some brokerages will may give you that for each $1MM you bring in, up to a limit). Do note that you’ll have to pay taxes on this incentive, and if they offer you a gift card it’s usually worse; you’ll only be able to spend the funds at the designated store, but it’s still a taxable benefit they’ll report to the IRS.

Multiple Brokerage Accounts

It is unwise to keep all your eggs in one basket, but brokerages may be an exception.

These entities are highly regulated and share their books with very reputable accounting/auditing firms. They carry tons of insurance, including $500k/account from the government’s SIPC as of 2018. Vanguard, for example, has a $250MM aggregate insurance policy through Lloyd’s of London as of 2018. A single brokerage also makes it easier to manage record-keeping and rebalancing. Most importantly, the big brokerages (Vanguard, Fidelity, Schwab, BofA/ML, etc) are arguably too big for Uncle Sam to let fail, and if that happened the laws are clear around bankruptcy proceedings for assets held in custody vs assets on the balance sheet of a financial institution (an important distinction).

The counterpoint is that if your brokerage pulls some Bernie Madoff-style book-cooking, there’s a tiny chance you could lose everything. Even if the government bailed you out, they might cap the bailout (say at $500k worth of assets) and you’d eat any losses over that.

I currently use one brokerage, but I may consider splitting assets in the future. It might even make sense to use a brokerage in another country to be insulated from issues affecting the US. That would add enormous complexity and also require an FBAR filing each year. You may also find that foreign brokerages don’t accept US citizens and/or charge exorbitant fees. If you hold foreign mutual funds in that account, you could be subject to Passive Foreign Investment Company (PFIC) taxes, which has a host of issues.

Rebalancing

As some of your investments do (relatively) well and others do (relatively) poorly, you could find yourself in a situation where your original asset allocation has drifted.

If you rebalance constantly you’ll waste lots of time (and transaction fees), and if you never do it then you’re not managing your risk appropriately. I recommend rebalancing semi-annually.

It’s best to avoid selling winning positions as that’s a taxable event, and instead just top-up your under-performers with post-tax dollars (see the section rebalance through non-reinvestment of dividends above for more).

FDIC

The Federal Deposit Insurance Corporation insures cash in your bank account for up to $250k in losses as of 2018. While this may encourage risk-taking and decrease stability in the financial system as a whole, it’s still useful for individuals.

You probably shouldn’t hold more than $250k in a bank account anyway, but if you need to then a “Deposit Sweep Program” may be ideal for you. You can have an account with one bank that partners with other FDIC insured banks to spread your funds across multiple institutions. Fidelity offers a free program like this that can effectively give you $1.25 MM of FDIC coverage as of 2018. It’s particularly useful when you need to pay a tax bill > $250k and don’t want the money to sit in a checking account uninsured while the funds are in transit (if paying by check the government can sometimes take weeks to cash it).

Trust and Integrity

In the investing world, it often feels like everyone is out to get you. As Legendary investor Warren Buffet told his investors in February 1988:

As they say in poker, “If you’ve been in the game 30 minutes and you don’t know who the patsy is, you’re the patsy.”

Everywhere you go you will cross paths with people trying to sell you something, and often they’re getting something in return: fees, commissions, kickbacks, unloading a position, etc. They’ll dazzle you with their understanding of complex issues you’ve never heard of, fancy terms you didn’t know existed, and talk of what “sophisticated investors” are doing. Fear sells.

When an investor loses their money (or just underperforms), whether it’s due to an outright scam or incompetence is a distinction without a difference. Value is hard to determine, but incentives are much easier to understand and often quite underrated. I refuse to invest in anything where incentives are not well aligned. The other party will always claim that the reason is an unexpected corner case, will never happen, is a “standard” term, etc. You’ll sleep better at night (and avoid downside risk) if you can ignore the FOMO on these deals.

Some advisors charge a flat (or percentage of AUM) fee, without any fees for performance. Some may consider this good (because they have no incentive to take extra risks with your money), or bad (because it limits their upside and thus may not attract the best performers). These advisors are often fiduciaries (i.e. required by law to act with your interests in mind), but commission-only advisors are not fiduciaries and must only propose investments that are “suitable” (a much lower standard). A popular technology-based example of a fiduciary is Personal Capital, though this model has been around long before them.

Life Insurance

Insurance against death (strangely known as “life insurance”) provides a guaranteed payout to your loved ones should something happen to you. Life insurance makes a lot of sense for a breadwinner without large savings. A young doctor with a family would be a perfect candidate, as they might have more medical school debt than assets but enough income to buy life insurance.

One tricky thing aspect is that a good life insurance policy typically requires access to your full medical history and some blood tests. If you’re diagnosed with terminal cancer today, you won’t be able to buy life insurance tomorrow. Also, if you get a curable form of cancer and are later fully cured (perhaps through surgery and/or chemotherapy), insurers will often still refuse to take on the risk of insuring you. If you’re worried about this, a creative solution is to buy whole life insurance for your children when they turn 18. The premiums will be very cheap (the odds that a teenager dies are quite low), so it’s a way to guarantee in advance that your children will have life insurance when they start their own family. In fact, you can even purchase a “guaranteed insurability option” that allows you to increase the size of your death-benefit later (perhaps as your family grows) without a medical exam.

Longevity Annuities

While life insurance can insure you against death, what financial protection do you have if you live to be 110 years old?

Longevity annuities are a financial product that pays out while you’re alive to collect the benefits. One strange side effect is that this may actually cause you to live longer. Be mindful of who you purchase from as there is counterparty risk in this transaction (and during retirement you can least afford any losses). The fees can also be considerable, so longevity annuities make less sense the more money you have; if you’re wealthy enough you can “self insure” and risk having less money leftover for your heirs if needed.

Revocable and Irrevocable Trusts

Trusts have a grantor (the person who creates the trust and contributes the initial assets), a trustee (a fiduciary who approves purchases and takes care of administrative record-keeping), and a beneficiary (the person for whom the trust is designed to help). Parents often use trusts to gift money to their child and structure them so that the child gets access to more funds as they age. In the meantime, the trustee approves spending of funds (at their discretion) for things like education, rent, and transportation. One bizarre aspect about our litigious society is that if a young beneficiary requests and successfully receives funds for frivolous purposes (say to start a business), this trust fund baby may later sue the trustee for breaching their fiduciary duty and allowing the beneficiary to squander their future funds. You can understand why trustees might be cautious to approve expenses!

When you create an irrevocable trust, it means that the beneficiary receives the money and the gift cannot be undone. If you are later sued, these previously gifted assets typically cannot be taken back from the beneficiary’s trust. So if you plan to give money to your heirs, you can use an irrevocable trust to guarantee that the money will make it to them. Be mindful that irrevocable really means irrevocable. Should you later have a need for the funds or a falling out with your beneficiary, you can’t undo the gift. Also, this gift will count toward your estate tax should it be large enough to qualify.

A revocable trust, however, can be reversed at any time. A typical structure is for the grantor to also be the trustee and beneficiary, and to specify other beneficiaries in the event of the grantor’s death (the revocable trust would then automatically convert to an irrevocable trust). Since the grantor is still the beneficiary in a revocable trust, asset transfers to the trust are non-taxable events. The trust will even use the same social security number as the grantor.

Trusts can own property, invest in hedge funds, buy companies, and open bank/brokerage accounts. Another benefit is that they already specify what happens to your assets when you die. Your heirs won’t need endure a public and lengthy probate process in court in order to divvy up your assets. For example, if the new trustee of the trust can prove to the bank that she is now the rightful trustee, she can sign a check on behalf of the trust to disburse the assets. I keep my assets in a revocable trust.

Frugality

Many wealthy people got that way through careful money management and strict spending habits. At a certain level of wealth, optimizing for savings (instead of income) becomes “penny-wise and pound foolish”. I’m very proud of my frugality, but for many successful people it’s important to keep in mind that a 50% reduction in costs might be quite challenging, while a 50% increase in earnings might be much more attainable (and repeatable).

As you get closer to the end of your life, you may be reminded of this wisdom:

Fly first class, or else your children will.

You can’t take it with you

Full disclosure: I very rarely fly first class, and I have no children.

Credit Card Rewards

Things to look for in a credit card:

  • $0 annual fee. Many cards will waive the fee in their first year, but then the price increases after that. The other end of the spectrum are cards like American Express Platinum and Chase Sapphire Reserve that are expensive but provide tons of benefits (that may or may not be worth it to you).
  • High cash-back. Unless you’re trying to optimize everything around discounted first-class international flights, airline and other company-specific rewards cards are rarely worth the effort. Cash is king, and a card that automatically pays you each month for using it usually provides a much better deal since you can use that cash wherever you like.
  • No foreign transaction fees. Many cards charge up to 3% to transact in another country!
  • A generous signup bonus. This is a one-time benefit (and usually only applies to cards with annual fees), but these promotions can be generous.

Every time you sign up for a new card you incur switching costs and (slightly) lower your credit score. It’s probably not worth the hassle to switch all the time. Websites like The Points Guy provide great analysis of what card is best for you.

I recommend privacy.com for managing your recurring subscription payments and making online purchases in general. Having your credit card number cloned, getting a new card, and updating your payment info everywhere is a common pain that can be easily eliminated. With privacy.com, you can create a virtual credit card number for each merchant/transaction. Full disclosure: I’m a small investor in the company.

Love What You Do and Never Work a Day in Your Life

It is often foolish advice to “follow your passion”, as the salaries of Art History majors are notoriously low. The Japanese have a concept of Ikigai, which is much more useful:

(Source)

An important consideration is that you don’t know how long you’ll live, and unexpected events and deteriorating health could cause your costs (or “burn rate”) to skyrocket. One way to minimize this risk is to never actually retire, a concept that is dreaded for some and stimulating for others.

Conclusion

Wealth management is a hard task with serious consequences, so it makes sense that an army of advisors and brokers have sprung up to sell every possible service.The goal of this post is to provide a high-level overview into many topics and a framework to more thoroughly evaluate these decisions (each of which could fill a whole book). You should view it as a checklist, and if anything here is new to you, I encourage you to spend time on more in-depth research. Because the cost-benefit analysis for each topic is unique to every individual, I’m unable to highlight which items are most important in general.

Acknowledgements

Thank you to the countless people who have given me investment advice over the years, and to the many friends who have helped me edit this post before publication. Special thanks to Adam Tzagournis for thorough review.

Beyond Wealth

There is more to life than making money, so once you’ve got that part under control I recommend checking out Whitney Tilson’s presentation The Five Calamities that Can Destroy Your Life and How to Avoid Them.

Outro

If everything works out, maybe I’ll see you on a boat one day:

Boats are definitely not a good investment

--

--