Too good to be true: the costs of trading

In this section of the introduction chapter, we will investigate a peculiar trading strategy that seems to outsmart the market while being very simple to execute. Of course, we will show why we fail to get rich easily right after we get our hopes up. But, as a consolidation, we will also look at a real investment firm that also tried to exploit this seeming loophole, with sobering semi-success.

Adjusted open prices

But let us start at the very beginning. We aim to beat the market or at least suffer less drawdowns by actively trading one of the most liquid assets of all, the one that we've been dealing with this whole chapter, the ETF investing in the S&P500 stock index, SPY. Again, we start by loading the daily price data:

import yfinance as yf
data = yf.download(["SPY"], period="max")

data

             Adj Close       Close  ...        Open    Volume
                 SPY         SPY  ...         SPY       SPY
Date                                ...                      
1993-01-29   24.684097   43.937500  ...   43.968750   1003200
1993-02-01   24.859665   44.250000  ...   43.968750    480500
1993-02-02   24.912329   44.343750  ...   44.218750    201300
1993-02-03   25.175673   44.812500  ...   44.406250    529400
1993-02-04   25.281013   45.000000  ...   44.968750    531500
...                ...         ...  ...         ...       ...
2024-07-25  538.409973  538.409973  ...  541.349976  61158300
2024-07-26  544.440002  544.440002  ...  542.280029  53763800
2024-07-29  544.760010  544.760010  ...  546.020020  39515800
2024-07-30  542.000000  542.000000  ...  546.260010  46487100
2024-07-31  550.799988  550.799988  ...  548.979980  16644479

[7932 rows x 6 columns]

Looking at that dataframe of price data, you might notice that it only contains an adjusted close price, but not an adjusted open price, or adjusted high/low prices. For the trading strategy we want to test here, we will not only adjust our position at the market close, but also on the market open. To avoid comparing apples to oranges, we also need to adjust the open prices for splits and dividend payments. To do that, we need to extract the factor that converts the raw close price into the adjusted close price. We can do this by dividing the adjusted close column by the respective values in the raw close column. Then we take the raw open price column and multiply it with this adjustment factor:

adj_factor = data['Adj Close']/data['Close']

adj_close = data['Adj Close']
adj_open = data['Open']*adj_factor
adj_open

                   SPY
Date                  
1993-01-29   24.701654
1993-02-01   24.701659
1993-02-02   24.842104
1993-02-03   24.947441
1993-02-04   25.263457
...                ...
2024-07-25  541.349976
2024-07-26  542.280029
2024-07-29  546.020020
2024-07-30  546.260010
2024-07-31  548.979980

[7932 rows x 1 columns]

Plotting the adjusted open and the adjusted close on top of each other, we barely see any difference, they overlap completely. This is expected, as any daily return (the difference between close an open) is tiny compared to the cumulative return over many years:

plt.plot(adj_open)
plt.plot(adj_close)

[<matplotlib.lines.Line2D object at 0x5e140a8>]

Note: Draw a rectangle with our mouse in the plot above to zoom in on the chart, then you will see that there are indeed small deviations between the open and close prices; use a right click to zoom out again.

Are stocks nocturnal animals?

Since we have two price series now, containing the open and the close price of each day, we can do a test that will sound utterly stupid at first: We want to ask during which period of the day the stock market actually generates profits and losses. If you think that stock markets generate profits during the actual trading hours, when people are actually trading, when information is passed around and when the brutal capitalistic process of competition mercilessly finds the optimal price of an asset -- then you are in for a surprise!

To answer our question of when during the day stock profits are made, we compute three different equity curves that accumulate different kinds of returns:

1. Open-to-close (OTC) returns: These returns capture the price changes that happen during the opening hours of the New York Stock Exchange (9:30am to 4:00pm). If we want to isolate and realize these returns, we need to buy at the market open each day, and sell at the market close each day.
2. Close-to-open (CTO) returns: These returns capture the price changes during the time in which the exchange is closed. But wait, how can prices change when no trading is allowed? Well, apart from extended trading hours in which some market participants are allowed to trade longer, asset prices can (and almost always do) show a gap between the last close price and the next open price. This gap summarizes all the information that became available overnight and is incorporated in the price upon market open with the first trades that are executed via a special opening auction. If we want to isolate and realize these returns, we need to buy at the market open each day, and sell at the market close each day.
3. Close-to-close (CTC) returns: These returns are the ones we have used thus far in this chapter, they represent us buying and holding an asset when accumulated into equity curve.

Which of these returns do you think make up for most of the profits (or losses)? Let's have a look. In the Python code snippet below, we calculate cumulative returns over time for all three cases using a mathematical trick: we first calculate the logarithm of our prices. From these logarithmic prices, we can calculate log-returns by subtracting two log-prices from each other. Log-returns have the benefit that we do not need to multiply them to account for compounding, we only need to sum them up. By building the cumulative sum over the log-returns, and applying the exponential function, we directly obtain the equity curve of investing 1$ in one of our trading strategies:

import numpy as np

# compute log prices
log_open_prices = np.log(adj_open)
log_close_prices = np.log(adj_close)

# compute log-returns for all strategies
OTC_log_returns = log_close_prices-log_open_prices
CTO_log_returns = log_open_prices-log_close_prices.shift(1)
CTC_log_returns = log_close_prices-log_close_prices.shift(1)

# convert to equity curves and plot
plt.plot(np.exp(np.cumsum(CTC_log_returns)), label='CTC')
plt.plot(np.exp(np.cumsum(OTC_log_returns)), label='OTC')
plt.plot(np.exp(np.cumsum(CTO_log_returns)), label='CTO')

plt.axhline(1, lw=0.7, c='C4', ls='--')
plt.ylabel('Portfolio value (USD)')
plt.yscale('log')

Wow! Almost no returns from open to close - that means stocks do not gain any significant value while the market is open! All the appreciation in value happens during the time when the market is closed - overnight! This "night effect" has been studied well in the literature and affects not just the US stock market, but markets worldwide. One theory is that holding stocks overnight poses a higher risk, e.g. due to earnings reports being published after the market closes. Since you cannot sell in the event of catastrophic news, the risk of holding overnight is higher compared to holding during active trading hours, and investors are compensated for taking risk. In an efficient market, the market structure should reward the overnight holders to a larger extent compared to daytime holders.

Looking at the chart above, you can almost completely avoid the drawdown during the Great Financial Crisis of 2008/2009 by not holding a position in stocks when the market is open. You buy at the close, sell again when the market opens the next day, and reduce your risk drastically. Simple as that.

Why are you not rich then?

Of course, all that glitters is not gold, and we will shortly see that exploiting the night effect is not as easy as it seems. To learn that lesson without losing money, we will again simulate an equity curve of investing in SPY from close to the next open every day, but this time we will consider transaction costs. This is an active trading strategy after all, not a buy-and-hold scenario in which we trade once and forget about the position.

In general, we have to consider three types of costs when doing a trade:

1. Transaction fees: Your broker will most likely charge you a fee (also called commissions) for executing the order. This fee can be a flat fee of e.g. $5 (like some retail-brokers do) for the whole order, irrespective of the size of the order (within some bounds), or a certain fee per share, or a certain percentage of the order value. Many retail brokers also offer a no-fee policy for very liquid products as long as one does not move around too much money.
2. The bid-ask spread: While stock info websites mostly print a single price on top of a stock chart (e.g. the price of the most recent transaction), there are always two prices: the bid price and the ask price. If you want to buy a stock right now, you can get it at the bid price, which is slightly higher than the ask price. If you want to sell a stock right now, you can do so at the ask price, which is slightly lower than the bid price. The difference between the ask and the bid price is often called the bid-ask spread. If you buy a stock and immediately sell the stock again, you lose the bid-ask spread. But who gets this money? The bid-ask spread that you lose goes to market makers, i.e. market participants who offer to trade by submitting limit orders that other traders can take by submitting a market order. Since market makers expose themselves to risk by guaranteeing to trade at a certain price with their limit orders, they should be compensated for this risk, and they ensure exactly that by always offering to sell at a higher price than they offer to buy a stock (market makers can and do lose money, though, if too many informed traders move the price). For very liquid ETFs such as SPY, the spread will usually stay well within 0.1% (or 10 basis points, bps.).
3. Slippage: If you execute a large order to buy or sell stocks right now, you may not be able to execute the complete order at the best bid price or best ask price anymore. Since the bid/ask prices are at any given moment just the limit orders with the best price that someone submitted, they have limited capacity, or volume. For example, the trader who submitted the limit order that is currently the best bid price may only want to sell 500 shares, but you maybe want to buy 1000 shares. In that case, you only get 500 shares at the best bid price, while the remaining 500 shares of your market order will be matched with other limit orders that offer a slightly worse price. This movement of the price against you as your market order eats up multiple limit orders is called "slippage". We will not care about slippage in this example for two reasons: first, to simulate slippage, we would need to have information on all the different limit orders that were present at the past times at which we want to trade in our simulation. This data is called "level-2 order book data" and is usually quite expensive to get from exchanges (except for most cypto-exchanges that usually offer this kind of data for free). Second, slippage only affects very large orders or orders in very illiquid markets, and we want to trade one of the most liquid products on earth.

To get a better idea of how trading costs affect our magical strategy of buying SPY at the close, and selling it again at the next open, we run a simulation of the equity curve, this time subtracting trading costs from each daily return. Note that this calculation is more intuitive when using percentage returns insetad of log-returns. First, let's convert the CTO log-returns to percentage returns:

CTO_returns = np.exp(CTO_log_returns) - 1
CTO_returns

                 SPY
Date                
1993-01-29       NaN
1993-02-01  0.000711
1993-02-02 -0.000706
1993-02-03  0.001409
1993-02-04  0.003487
...              ...
2024-07-25  0.000222
2024-07-26  0.007188
2024-07-29  0.002902
2024-07-30  0.002754
2024-07-31  0.012878

[7932 rows x 1 columns]

Next, we subtract our assumed daily trading costs from each daily return. Let's start with the assumption that we can get in and out of the trade at 0.05% cost, or 5 basis points (bps). Then we get the following cost-adjusted returns:

CTO_returns_cost_adj = CTO_returns - 5/10_000
CTO_returns_cost_adj

                 SPY
Date                
1993-01-29       NaN
1993-02-01  0.000211
1993-02-02 -0.001206
1993-02-03  0.000909
1993-02-04  0.002987
...              ...
2024-07-25 -0.000278
2024-07-26  0.006688
2024-07-29  0.002402
2024-07-30  0.002254
2024-07-31  0.012378

[7932 rows x 1 columns]

Finally, we reconstruct the equity curves from the percentage returns and plot both the raw and the cost-adjusted curves:

plt.plot(np.cumprod(CTO_returns+1), label='no cost')
plt.plot(np.cumprod(CTO_returns_cost_adj+1), label='5 bps cost')
plt.yscale('log')

Ouch! Even the low trading costs of 5 bps per day completely obliterate any benefit that the night effect may hold. Let's assume that we can instead trade with only 1 bps cost each day:

CTO_returns_cost_adj2 = CTO_returns - 1/10_000

plt.plot(np.cumprod(CTO_returns+1), label='no cost')
plt.plot(np.cumprod(CTO_returns_cost_adj+1), label='5 bps cost')
plt.plot(np.cumprod(CTO_returns_cost_adj2+1), label='1 bps cost')
plt.yscale('log')

Even just 1 bps trading cost reduces the returns by quite a bit, and makes the equity curve lag behind a buy-and-hold investment in SPY (which yielded about the same return as the no-cost simulation above, but with larger drawdowns in-between). In the next section and in Chapter 3 we will discuss further details of how to handle trading costs, including withdrawals as a special kind of trading cost that can considerably alter portfolio selection, e.g. if you plan to live off of your investments and regularly need to withdraw a fraction of your profits!

At this point, you may think that there is no way to exploit the night effect, as any sort of trading cost will immediately eat up the excess returns, and more. But actually, the idea of exploiting the the night still lives on, and up until recently, there was even an ETF that you could buy that implemented the kind of trading strategy simulated above. But how did they tackle the problem of trading costs? Let's have a brief look!

More than a thought experiment

While we merely used the night-effect as an example to showcase the effect of trading costs, others have actually built products around this effect that investors could easily invest in! For a relatively short period from 2022 until 2023, the investment firm NightShares operated the ETFs NSPY and NIWM that aimed at exploiting the night effect in the S&P500 and the Russell2000 index, respectively. To reduce trading costs, NightShares ETFs did not buy and sell the actual stocks within the stock indices or an index ETF each day, but instead they traded futures and swap products. These products are contracts about the right to buy or sell a certain underlying financial asset (like stock indices) at a predefined time.

Futures and swap contracts allow traders to easily enter short positions (i.e. bet on falling prices without first borrowing shares to sell), to enter leveraged positions (i.e. get a x-fold return of the underlying asset), and to trade large positions at much lower costs compared to trading the asset itself. In Chapter 5, we will take a closer look at active futures trading strategies on crypto markets as crypto futures markets are much more easily accessible for retail traders, whereas trading strategies that operate on traditional futures markets need a starting capital of at least $100,000. For now, we just want to have a closer look at the performance of the night-effect ETF NSPY and roughly estimate the effective trading costs that NightShares still had to manage. As NSPY is delisted and not traded anymore by now, the data is not available from Yahoo Finance, so we use data from a data vendor that also offers historial data of delisted symbols (It is actually very important to include delisted products in certain portfolio simulations, as one otherwise introduces significant bias as companies that are still around today are by definition more successful than companies that are not around anymore). Let's load the data:

import pandas as pd

df = pd.read_csv('data/NSPY.csv', parse_dates=True, index_col='date')

df

              open    high      low    close  adjusted_close  volume
date                                                                
2022-06-28  31.750  31.750  31.5640  31.6300         31.3015   46972
2022-06-29  33.210  34.970  31.6100  31.6900         31.3609   66605
2022-06-30  31.690  31.690  31.2004  31.2900         30.9650   35350
2022-07-01  31.270  31.350  31.2500  31.2995         30.9744   61999
2022-07-05  31.200  31.200  30.8100  30.8900         30.5692   26161
...            ...     ...      ...      ...             ...     ...
2023-08-09  29.555  29.555  29.5550  29.5550         29.5550       0
2023-08-10  29.555  29.555  29.5550  29.5550         29.5550       0
2023-08-11  29.555  29.555  29.5550  29.5550         29.5550       0
2023-08-14  29.555  29.555  29.5550  29.5550         29.5550       0
2023-08-15  29.555  29.555  29.5550  29.5550         29.5550       0

[285 rows x 6 columns]

You may notice that the last few rows show a volume of zero, corresponding to days when trading the ETF had already ceased, so we crop the data and only keep the time period of active trading (until the end of July 2023):

df = df.loc[:'2023-07-31']
df

               open     high      low    close  adjusted_close  volume
date                                                                  
2022-06-28  31.7500  31.7500  31.5640  31.6300         31.3015   46972
2022-06-29  33.2100  34.9700  31.6100  31.6900         31.3609   66605
2022-06-30  31.6900  31.6900  31.2004  31.2900         30.9650   35350
2022-07-01  31.2700  31.3500  31.2500  31.2995         30.9744   61999
2022-07-05  31.2000  31.2000  30.8100  30.8900         30.5692   26161
...             ...      ...      ...      ...             ...     ...
2023-07-25  29.2497  29.2497  29.2200  29.2300         29.2300    4993
2023-07-26  29.1700  29.2150  29.1700  29.2150         29.2150     760
2023-07-27  28.5700  29.4506  28.5700  29.3600         29.3600   21384
2023-07-28  29.4800  29.5100  29.4800  29.5100         29.5100    1155
2023-07-31  29.5100  29.5550  29.5100  29.5550         29.5550     973

[274 rows x 6 columns]

To investigate the performance of NSPY, we calculate three different equity curves:

1. Buy and hold NSPY: We invest in the NightShares ETF on the first day that it was tradable (end of June 2022), and hold it until it got delisted (end of July 2023). We do this by first computing daily percentage returns and then accumulating them using the function np.cumprod, as we have done before. The resulting equity curve start at 1 and shows the time course of the value of a portfolio started with $1 invested in NSPY.
2. Buy and hold SPY: The classic benchmark that will show us whether NSPY is actually successful in exploiting the night effect. We compute the equity curve by starting with the CTC_log_returns series from before, convert them to daily percentage returns to be consistent with the calculation for NSPY, and accumulate the returns over time.
3. Zero-fee close-to-open SPY: As a second benchmark, we assume that we can buy SPY at the close and sell it again at the next open without any trading costs. While - as we have seen before - this is rather unrealistic, this benchmark will show us how close NightShares came to this optimal trade execution by using futures and swaps.

# daily percentage returns of NSPY
NSPY_returns = df['adjusted_close'].pct_change().dropna()

# Close-to-close returns of SPY within the same period
CTC_returns = np.exp(CTC_log_returns) - 1
CTC_returns_cropped = CTC_returns.loc[NSPY_returns.index]

# Close-to-open returns of SPY within the same period
CTO_returns_cropped = CTO_returns.loc[NSPY_returns.index]

plt.plot(np.cumprod(NSPY_returns + 1), label='NSPY')
plt.plot(np.cumprod(CTC_returns_cropped + 1), label='CTC')
plt.plot(np.cumprod(CTO_returns_cropped + 1), label='CTO')
plt.ylabel('Portfolio value');

The results are actually surprising in different ways: first, the night effect substantially underperformed the classic buy-and-hold strategy during the year that the NightShares ETF existed. And this actually fits the comparison plots we have created in previous sections, the night effect (i.e. the close-to-open returns) does not always outfperform the classic close-to-close returns, there are periods when it underperforms. If we subtract the daily close-to-close returns from the daily close-to-open returns, and smooth these values by a rolling window over the previous 252 trading days (one year), we can identify periods of outperformance (positive values) and underperformance (negative values) of NSPY:

diff = (CTO_returns - CTC_returns).rolling(252).mean()*100

plt.plot(diff)
plt.axhline(0, lw=0.7, c='C4', ls='--')
plt.ylabel('Mean daily outperformance (%)')

Text(0, 0.5, 'Mean daily outperformance (%)')

As we can see, NSPY mainly outperforms under challenging market conditions, for example during the dot-com bubble and the financial crisis of 2008, but otherwise often slightly underperforms. The fact that NightShares created this ETF mid 2022 thus was bad luck, as investors were not convinced looking at the short track record and the substantial underperformance. Had NSPY been created before the financial crisis of 2008, then investors probably would have seen its value in reducing risk during stock market crashes!

The second surprising result is the apparent trading efficiency of NSPY: the equity curve almost perfectly overlaps with the zero-fee simulation of trading the night effect! To get a better idea of the minimal trading costs that NightShares managed to get for their daily position changes, we can plot simulations ranging from 0.25 bps to 1 bps and compare them to the performance of NSPY:

bps_0_25 = CTO_returns_cropped - 0.25/10_000
bps_0_50 = CTO_returns_cropped - 0.50/10_000
bps_0_75 = CTO_returns_cropped - 0.75/10_000
bps_1_00 = CTO_returns_cropped - 1.00/10_000

plt.plot(np.cumprod(NSPY_returns + 1), label='NSPY')
plt.plot(np.cumprod(bps_0_25 + 1), label='0.25 bps')
plt.plot(np.cumprod(bps_0_50 + 1), label='0.50 bps')
plt.plot(np.cumprod(bps_0_75 + 1), label='0.75 bps')
plt.plot(np.cumprod(bps_1_00 + 1), label='1.00 bps')
plt.ylabel('Portfolio value');

By zooming in on the end of the series, we see that the equity curve of NSPY aligns best with simulated costs of 0.75 bps, whereas simulated costs of 0.25 bps and 0.50 bps slightly outperform NSPY, and simulated costs of 1.00 bps underperform NSPY. Pretty impressive!

The graph illustrates how using derivative assets such as futures and swaps can help to bring down trading costs for some kind of strategies. We will explore this topic further in Chapter 5, where we build our own futures trading strategy using crypto perpetual futures contracts (not because we firmly believe in the future of crypto, but because these products allow small traders to start trading with a very small bankroll and get high-quality data for free - the exact opposite of traditional commodity futures markets).

In the next section, we will have a closer look at other ETFs that use futures and swaps behind the scenes to allow retail investors to invest in leveraged assets, i.e. to gain (or lose) multiple times the daily return of stock indices and other asset classes. What could possibly go wrong?