Constructing an SME Composite Index for the Hungarian Economy

Overview

This section documents the construction, comparison, and final selection of the SME composite index. After transforming the deflated indicators into annual growth rates, several weighting methods are evaluated in order to assess how different aggregation choices affect the resulting index. The aim is to select a specification that is statistically robust, economically interpretable, and stable over time.

The analysis proceeds in three steps. First, the indicator weights implied by the alternative methods are compared. Second, the resulting composite indices are examined jointly to evaluate whether they capture a similar SME cycle. Finally, a leave-one-out robustness analysis is used to assess how sensitive each method is to the omission of individual indicators.

Data input, deflation and growth-rate transformation

First, we load the SME indicator database from an Excel file. The dataset contains annual observations by firm size class and by indicator. Each observation includes the year, the SME size category, the name of the variable, and the corresponding deflated value.

Before calculating growth rates, nominal monetary indicators were converted into real terms using indicator-specific deflators. Indicators measured in physical or count units were not deflated, since they already represent quantities rather than monetary values.

No.	Indicator	Deflator used
1	Change in the number of active enterprises	Not deflated
2	Change in the number of employed persons	Not deflated
3	Volume change in personnel expenditures	Consumer price index
4	Output volume change based on enterprise turnover	GDP implicit price deflator
5	Volume change in value added	GDP implicit price deflator
6	Export volume change based on export revenues	Export implicit price index from national accounts
7	Volume change in gross investment	Investment price index
8	Volume change in R&D expenditures	Service price index

The dataset is originally stored in long format, where the different indicators appear as values of a single variable column. Therefore, the data are reshaped into wide format using the year and the firm size class as panel identifiers. After this transformation, each indicator appears as a separate column.

Finally, year-on-year percentage changes are calculated separately within each firm size class. This is important because growth rates must always compare an observation to the previous year within the same SME category, rather than to observations from another size class. Rows where all indicator growth rates are missing are removed from the final growth-rate dataset.

Python codeCell 1

Show code

import pandas as pd

kkv_deflated = pd.read_excel(r"[PATH_TO_EXCEL_FILE]", index_col = False)

wide = kkv_deflated.pivot_table(
    index=["year", "size_class"],
    columns="variable",
    values="value_deflated"
).reset_index()

indicator_cols = [
    col for col in wide.columns
    if col not in ["year", "size_class"]
]

wide = wide.sort_values(["size_class", "year"]).copy()

growth = wide.copy()

growth[indicator_cols] = (
    wide
    .groupby("size_class")[indicator_cols]
    .pct_change()
    * 100
)

growth = growth.dropna(subset=indicator_cols, how="all").copy()

Weighting methods used for the composite SME index

Let $g_{j,t}^{(s)}$ denote the annual percentage change of indicator $j$ in year $t$ and size class $s$, where $j=1,\ldots,J$. In this application, the index is based on $J=8$ indicators. For each method, the composite index is calculated as:

I_t^{(s)} = \sum_{j=1}^{J} w_j g_{j,t}^{(s)}.

1. Equal weighting

w_j^{RAW} = \frac{1}{J}.

This method assigns the same importance to all indicators and is used as the simplest benchmark specification.

2. Inverse-volatility weighting

First, the time-series volatility of each indicator is calculated:

\sigma_j = \sqrt{ \frac{1}{T-1} \sum_{t=1}^{T} \left(g_{j,t}-\bar{g}_j\right)^2 }.

The weights are then defined as:

w_j^{IV} = \frac{\sigma_j^{-1}} {\sum_{k=1}^{J}\sigma_k^{-1}}.

This method gives larger weights to more stable indicators and reduces the influence of highly volatile series.

3. PCA-based weighting

The indicators are first standardized:

z_{j,t} = \frac{g_{j,t}-\bar{g}_j}{\sigma_j}.

The first principal component is:

PC_{1,t} = \sum_{j=1}^{J} a_j z_{j,t}.

The PCA weights are calculated from the squared loadings:

w_j^{PCA} = \frac{a_j^2} {\sum_{k=1}^{J} a_k^2}.

This method gives larger weights to indicators that contribute more strongly to the common variation captured by the first principal component.

4. Factor-analysis weighting

The one-factor model is written as:

z_{j,t} = \lambda_j f_t + u_{j,t}.

The factor-analysis weights are calculated from the squared factor loadings:

w_j^{FA} = \frac{\lambda_j^2} {\sum_{k=1}^{J} \lambda_k^2}.

This method gives larger weights to indicators that are more strongly related to the estimated common SME factor.

5. PCA × inverse-volatility weighting

The hybrid PCA and inverse-volatility weight is obtained as:

w_j^{PCA \times IV} = \frac{ w_j^{PCA} w_j^{IV} }{ \sum_{k=1}^{J} w_k^{PCA} w_k^{IV} }.

Equivalently:

w_j^{PCA \times IV} = \frac{ a_j^2 \sigma_j^{-1} }{ \sum_{k=1}^{J} a_k^2 \sigma_k^{-1} }.

This method gives high weight only to indicators that are both informative for the common PCA component and relatively stable over time.

6. Factor analysis × inverse-volatility weighting

The hybrid factor-analysis and inverse-volatility weight is defined as:

w_j^{FA \times IV} = \frac{ w_j^{FA} w_j^{IV} }{ \sum_{k=1}^{J} w_k^{FA} w_k^{IV} }.

Equivalently:

w_j^{FA \times IV} = \frac{ \lambda_j^2 \sigma_j^{-1} }{ \sum_{k=1}^{J} \lambda_k^2 \sigma_k^{-1} }.

This method gives high weight to indicators that are strongly connected to the common latent factor while also having relatively low volatility.

Python codeCell 2

Show code

import numpy as np
import pandas as pd
from sklearn.decomposition import PCA, FactorAnalysis

column_rename_map = {
    "Bruttó beruházás Mrd Ft": "Gross investment, HUF billion",
    "Export árbevétel Mrd Ft": "Export revenue, HUF billion",
    "Foglalkoztatottak száma fő": "Employment, persons",
    "Hozzáadott érték Mrd Ft": "Value added, HUF billion",
    "K+F Mrd Ft": "R&D, HUF billion",
    "Személyi jellegű ráfordítások Mrd Ft": "Personnel costs, HUF billion",
    "Árbevétel Mrd FT": "Revenue, HUF billion",
    "Árbevétel Mrd Ft": "Revenue, HUF billion",
    "Működő válallkozások száma (akinek van árbevétele)": "Active enterprises with revenue",
    "Működő vállalkozások száma (akinek van árbevétele)": "Active enterprises with revenue"
}

growth = growth.rename(columns=column_rename_map)

if "wide" in globals():
    wide = wide.rename(columns=column_rename_map)

gross_investment_col = "Gross investment, HUF billion"
export_revenue_col = "Export revenue, HUF billion"
employment_col = "Employment, persons"
value_added_col = "Value added, HUF billion"
rd_col = "R&D, HUF billion"
active_enterprises_col = "Active enterprises with revenue"
personnel_cost_col = "Personnel costs, HUF billion"
revenue_col = "Revenue, HUF billion"

selected_indicators = [
    gross_investment_col,
    export_revenue_col,
    employment_col,
    value_added_col,
    rd_col,
    active_enterprises_col,
    personnel_cost_col,
    revenue_col
]

missing_cols = [col for col in selected_indicators if col not in growth.columns]

if len(missing_cols) > 0:
    raise ValueError(f"The following indicator columns are missing: {missing_cols}")

kkv_total_growth = (
    growth[growth["size_class"] == "KKV összesen"]
    .sort_values("year")
    .copy()
)

kkv_total_growth[selected_indicators] = (
    kkv_total_growth[selected_indicators]
    .replace([np.inf, -np.inf], np.nan)
)

X = kkv_total_growth[selected_indicators].copy()

def normalize_weights(weights, index):
    s = pd.Series(weights).reindex(index).astype(float).fillna(0)
    total = s.sum()

    if pd.isna(total) or np.isclose(total, 0):
        s = pd.Series(1 / len(index), index=index)
    else:
        s = s / total

    return s

def prepare_standardized_matrix(X):
    mu = X.mean(axis=0)
    sigma = X.std(axis=0, ddof=1).replace(0, np.nan)

    Z = (X - mu) / sigma
    Z = Z.dropna(axis=1, how="all")
    Z = Z.loc[:, Z.std(axis=0, ddof=1) > 0]
    Z = Z.fillna(Z.mean(axis=0))

    return Z, X[Z.columns].copy()

def get_inverse_volatility_weights(X):
    volatility = X.std(axis=0, skipna=True, ddof=1).replace(0, np.nan)
    inverse_volatility = 1 / volatility
    weights = inverse_volatility / inverse_volatility.sum()

    return normalize_weights(weights, selected_indicators)

def get_pca_weights(X):
    Z, X_used = prepare_standardized_matrix(X)

    if Z.shape[1] >= 2:
        pca = PCA(n_components=1)
        score = pca.fit_transform(Z).flatten()

        loadings = pd.Series(
            pca.components_[0],
            index=Z.columns
        )

        reference_index = X_used.mean(axis=1, skipna=True).values
        correlation = np.corrcoef(score, reference_index)[0, 1]

        if pd.notna(correlation) and correlation < 0:
            loadings = -loadings

        weights = loadings ** 2
        weights = weights / weights.sum()

    else:
        weights = pd.Series(
            1 / X_used.shape[1],
            index=X_used.columns
        )

    return normalize_weights(weights, selected_indicators)

def get_factor_analysis_weights(X):
    Z, X_used = prepare_standardized_matrix(X)

    if Z.shape[1] >= 2:
        factor_model = FactorAnalysis(n_components=1, random_state=123)
        score = factor_model.fit_transform(Z).flatten()

        loadings = pd.Series(
            factor_model.components_[0],
            index=Z.columns
        )

        reference_index = X_used.mean(axis=1, skipna=True).values
        correlation = np.corrcoef(score, reference_index)[0, 1]

        if pd.notna(correlation) and correlation < 0:
            loadings = -loadings

        weights = loadings ** 2
        weights = weights / weights.sum()

    else:
        weights = pd.Series(
            1 / X_used.shape[1],
            index=X_used.columns
        )

    return normalize_weights(weights, selected_indicators)

raw_weights_final = pd.Series(
    1 / len(selected_indicators),
    index=selected_indicators
)

inverse_volatility_weights_final = get_inverse_volatility_weights(X)
pca_weights_final = get_pca_weights(X)
factor_analysis_weights_final = get_factor_analysis_weights(X)

pca_x_inverse_volatility_weights_final = normalize_weights(
    pca_weights_final * inverse_volatility_weights_final,
    selected_indicators
)

factor_x_inverse_volatility_weights_final = normalize_weights(
    factor_analysis_weights_final * inverse_volatility_weights_final,
    selected_indicators
)

weights_table = pd.DataFrame({
    "Indicator": selected_indicators,
    "RAW": raw_weights_final.values,
    "Inverse volatility": inverse_volatility_weights_final.values,
    "PCA": pca_weights_final.values,
    "Factor analysis": factor_analysis_weights_final.values,
    "PCA × inverse volatility": pca_x_inverse_volatility_weights_final.values,
    "Factor analysis × inverse volatility": factor_x_inverse_volatility_weights_final.values
})

weights_table = weights_table.set_index("Indicator")

weights_table_percent = weights_table * 100

display(weights_table)

Output

	RAW	Inverse volatility	PCA	Factor analysis	PCA × inverse volatility	Factor analysis × inverse volatility
Indicator
Gross investment, HUF billion	0.125	0.113929	0.070143	0.048511	0.057709	0.039305
Export revenue, HUF billion	0.125	0.123657	0.095844	0.081857	0.085588	0.071986
Employment, persons	0.125	0.168870	0.177104	0.200847	0.215978	0.241208
Value added, HUF billion	0.125	0.123228	0.159252	0.150011	0.141717	0.131464
R&D, HUF billion	0.125	0.052915	0.000092	0.001192	0.000035	0.000449
Active enterprises with revenue	0.125	0.101672	0.172292	0.201078	0.126501	0.145392
Personnel costs, HUF billion	0.125	0.203966	0.165119	0.180914	0.243212	0.262425
Revenue, HUF billion	0.125	0.111763	0.160154	0.135590	0.129260	0.107771

Interpretation of the weighting results

The weighting table shows that the six methods represent different aggregation logics. The RAW index is fully neutral, assigning each of the eight indicators the same weight of $12.5%$, while the other methods use either stability, common variation, or both to determine the weights. The inverse-volatility method gives more weight to stable indicators, whereas the PCA- and factor-analysis-based methods emphasize variables that are more strongly connected to the common SME dynamics.

The next figure compares the resulting SME composite indices over time. The four multivariate statistical methods, namely PCA, factor analysis, PCA × inverse volatility and factor analysis × inverse volatility, move more closely together, suggesting that they capture a similar underlying SME cycle. The RAW index shows the largest fluctuations, because equal weighting gives the same role to volatile indicators as to more stable ones.

Python codeCell 3

Show code

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

growth_plot = growth.copy()

growth_plot["size_class"] = growth_plot["size_class"].replace({
    "KKV összesen": "SME total",
    "0-9": "Micro",
    "10-49": "Small",
    "50-249-": "Medium"
})

sme_total_growth = (
    growth_plot[growth_plot["size_class"] == "SME total"]
    .sort_values("year")
    .copy()
)

sme_total_growth[selected_indicators] = (
    sme_total_growth[selected_indicators]
    .replace([np.inf, -np.inf], np.nan)
)

def weighted_average(df, cols, weights):
    cols = [col for col in cols if col in weights.index]

    if len(cols) == 0:
        return pd.Series(np.nan, index=df.index)

    w = weights[cols].copy()
    w = w / w.sum()

    x = df[cols].astype(float)

    values = []

    for _, row in x.iterrows():
        valid = row.notna()

        if valid.sum() == 0:
            values.append(np.nan)
        else:
            w_valid = w[valid]
            w_valid = w_valid / w_valid.sum()
            values.append(np.sum(row[valid] * w_valid))

    return pd.Series(values, index=df.index)

compare_df = pd.DataFrame({
    "Year": sme_total_growth["year"].values,
    "RAW": weighted_average(
        sme_total_growth,
        selected_indicators,
        raw_weights_final
    ).values,
    "Inverse volatility": weighted_average(
        sme_total_growth,
        selected_indicators,
        inverse_volatility_weights_final
    ).values,
    "PCA": weighted_average(
        sme_total_growth,
        selected_indicators,
        pca_weights_final
    ).values,
    "Factor analysis": weighted_average(
        sme_total_growth,
        selected_indicators,
        factor_analysis_weights_final
    ).values,
    "PCA × inverse volatility": weighted_average(
        sme_total_growth,
        selected_indicators,
        pca_x_inverse_volatility_weights_final
    ).values,
    "Factor analysis × inverse volatility": weighted_average(
        sme_total_growth,
        selected_indicators,
        factor_x_inverse_volatility_weights_final
    ).values
})


plt.figure(figsize=(12, 7))

for col in compare_df.columns[1:]:
    plt.plot(
        compare_df["Year"],
        compare_df[col],
        marker="o",
        linewidth=2,
        label=col
    )

plt.axhline(0, linestyle="--", linewidth=1)

plt.title("Comparison of composite SME indices by weighting method")
plt.xlabel("Year")
plt.ylabel("Annual percentage change index")
plt.grid(True, alpha=0.3)
plt.legend()
plt.tight_layout()
plt.show()

Output

Leave-one-out robustness analysis

The robustness of the composite indices is evaluated with a leave-one-out procedure. For each weighting method $m$, the full index $I_t^{(m)}$ is first calculated using all $J=8$ indicators. Then, each indicator is removed one at a time, the method-specific weights are re-estimated on the remaining $J-1$ indicators, and a new leave-one-out index $I_{t,-j}^{(m)}$ is constructed.

The first robustness measure is the correlation between the full index and the leave-one-out index:

R_j^{(m)} = \operatorname{corr} \left( I_t^{(m)}, I_{t,-j}^{(m)} \right).

The average correlation-based robustness score is:

S_{\mathrm{corr}}^{(m)} = \frac{1}{J} \sum_{j=1}^{J} R_j^{(m)}.

Higher values of $S_{\mathrm{corr}}^{(m)}$ indicate that the main time-series pattern of the index is preserved after removing individual indicators.

As a second measure, the mean squared deviation between the full and leave-one-out indices is calculated:

D_j^{(m)} = \frac{1}{T} \sum_{t=1}^{T} \left( I_t^{(m)} - I_{t,-j}^{(m)} \right)^2.

The average deviation-based robustness score is:

S_{\mathrm{MSE}}^{(m)} = \frac{1}{J} \sum_{j=1}^{J} D_j^{(m)}.

Lower values of $S_{\mathrm{MSE}}^{(m)}$ indicate that omitting one indicator changes the level and amplitude of the index only slightly. Thus, the correlation measure captures whether the time-series pattern is preserved, while the MSE measure captures the size of the deviation from the original index.

Python codeCell 4

Show code

import numpy as np
import pandas as pd
from sklearn.decomposition import PCA, FactorAnalysis

column_rename_map = {
    "Bruttó beruházás Mrd Ft": "Gross investment, HUF billion",
    "Export árbevétel Mrd Ft": "Export revenue, HUF billion",
    "Foglalkoztatottak száma fő": "Employment, persons",
    "Hozzáadott érték Mrd Ft": "Value added, HUF billion",
    "K+F Mrd Ft": "R&D, HUF billion",
    "Személyi jellegű ráfordítások Mrd Ft": "Personnel costs, HUF billion",
    "Árbevétel Mrd FT": "Revenue, HUF billion",
    "Árbevétel Mrd Ft": "Revenue, HUF billion",
    "Működő válallkozások száma (akinek van árbevétele)": "Active enterprises with revenue",
    "Működő vállalkozások száma (akinek van árbevétele)": "Active enterprises with revenue"
}

growth_robustness = growth.rename(columns=column_rename_map).copy()

selected_indicators = [
    "Gross investment, HUF billion",
    "Export revenue, HUF billion",
    "Employment, persons",
    "Value added, HUF billion",
    "R&D, HUF billion",
    "Active enterprises with revenue",
    "Personnel costs, HUF billion",
    "Revenue, HUF billion"
]

missing_cols = [col for col in selected_indicators if col not in growth_robustness.columns]

if len(missing_cols) > 0:
    raise ValueError(f"The following indicator columns are missing: {missing_cols}")

sme_total_growth = (
    growth_robustness[growth_robustness["size_class"].isin(["KKV összesen", "SME total"])]
    .sort_values("year")
    .copy()
)

sme_total_growth[selected_indicators] = (
    sme_total_growth[selected_indicators]
    .replace([np.inf, -np.inf], np.nan)
)

def standardize_for_statistical_weighting(df, cols):
    X = df[cols].copy()

    mu = X.mean(axis=0)
    sigma = X.std(axis=0, ddof=1).replace(0, np.nan)

    Z = (X - mu) / sigma
    Z = Z.dropna(axis=1, how="all")
    Z = Z.loc[:, Z.std(axis=0, ddof=1) > 0]
    Z = Z.fillna(Z.mean(axis=0))

    X_used = X[Z.columns].copy()

    return Z, X_used

def compute_weights(method, df, cols):
    cols = [col for col in cols if col in df.columns]

    if len(cols) == 0:
        raise ValueError("No usable indicators were found.")

    if method == "RAW":
        return pd.Series(1 / len(cols), index=cols, name="weight")

    if method == "Inverse volatility":
        vol = df[cols].std(axis=0, skipna=True, ddof=1).replace(0, np.nan)
        inv_vol = (1 / vol).replace([np.inf, -np.inf], np.nan).dropna()

        if len(inv_vol) == 0:
            return pd.Series(1 / len(cols), index=cols, name="weight")

        w = inv_vol / inv_vol.sum()
        w.name = "weight"

        return w

    if method == "PCA":
        Z, X_used = standardize_for_statistical_weighting(df, cols)

        if Z.shape[1] >= 2:
            pca = PCA(n_components=1)
            pca_score = pca.fit_transform(Z).flatten()

            loadings = pd.Series(pca.components_[0], index=Z.columns)

            simple_ref = X_used.mean(axis=1, skipna=True).values
            corr = np.corrcoef(pca_score, simple_ref)[0, 1]

            if pd.notna(corr) and corr < 0:
                loadings = -loadings

            w = loadings ** 2
            w = w / w.sum()
            w.name = "weight"

            return w

        return pd.Series(1 / X_used.shape[1], index=X_used.columns, name="weight")

    if method == "Factor analysis":
        Z, X_used = standardize_for_statistical_weighting(df, cols)

        if Z.shape[1] >= 2:
            fa = FactorAnalysis(n_components=1, random_state=123)
            factor_score = fa.fit_transform(Z).flatten()

            loadings = pd.Series(fa.components_[0], index=Z.columns)

            simple_ref = X_used.mean(axis=1, skipna=True).values
            corr = np.corrcoef(factor_score, simple_ref)[0, 1]

            if pd.notna(corr) and corr < 0:
                loadings = -loadings

            w = loadings ** 2

            if w.sum() == 0 or pd.isna(w.sum()):
                return pd.Series(1 / len(w), index=w.index, name="weight")

            w = w / w.sum()
            w.name = "weight"

            return w

        return pd.Series(1 / X_used.shape[1], index=X_used.columns, name="weight")

    if method == "PCA × inverse volatility":
        w_pca = compute_weights("PCA", df, cols)
        w_iv = compute_weights("Inverse volatility", df, list(w_pca.index))

        common_cols = w_pca.index.intersection(w_iv.index)
        raw = w_pca[common_cols] * w_iv[common_cols]

        w = raw / raw.sum()
        w.name = "weight"

        return w

    if method == "Factor × inverse volatility":
        w_factor = compute_weights("Factor analysis", df, cols)
        w_iv = compute_weights("Inverse volatility", df, list(w_factor.index))

        common_cols = w_factor.index.intersection(w_iv.index)
        raw = w_factor[common_cols] * w_iv[common_cols]

        w = raw / raw.sum()
        w.name = "weight"

        return w

    raise ValueError(f"Unknown method: {method}")

def weighted_index_series(df, weights):
    x = df[weights.index].astype(float)
    values = []

    for _, row in x.iterrows():
        valid = row.notna()

        if valid.sum() == 0:
            values.append(np.nan)
        else:
            w_valid = weights[valid]
            w_valid = w_valid / w_valid.sum()
            values.append(np.sum(row[valid] * w_valid))

    return pd.Series(values, index=df.index)

def safe_corr(x, y):
    tmp = pd.concat([x, y], axis=1).dropna()

    if tmp.shape[0] < 2:
        return np.nan

    if tmp.iloc[:, 0].std(ddof=1) == 0 or tmp.iloc[:, 1].std(ddof=1) == 0:
        return np.nan

    return tmp.iloc[:, 0].corr(tmp.iloc[:, 1])

def safe_mean_squared_deviation(x, y):
    tmp = pd.concat([x, y], axis=1).dropna()

    if tmp.shape[0] < 1:
        return np.nan

    diff = tmp.iloc[:, 0] - tmp.iloc[:, 1]

    return np.mean(diff ** 2)

methods = [
    "RAW",
    "Inverse volatility",
    "PCA",
    "Factor analysis",
    "PCA × inverse volatility",
    "Factor × inverse volatility"
]

method_latex_names = {
    "RAW": "RAW",
    "Inverse volatility": "Inverse volatility",
    "PCA": "PCA",
    "Factor analysis": "Factor analysis",
    "PCA × inverse volatility": r"PCA $\times$ inverse volatility",
    "Factor × inverse volatility": r"Factor $\times$ inverse volatility"
}

robustness_summary = []

for method in methods:
    full_weights = compute_weights(
        method=method,
        df=sme_total_growth,
        cols=selected_indicators
    )

    full_index = weighted_index_series(
        df=sme_total_growth,
        weights=full_weights
    )

    loo_corrs = []
    loo_msds = []

    for dropped_indicator in selected_indicators:
        loo_cols = [
            col for col in selected_indicators
            if col != dropped_indicator
        ]

        loo_weights = compute_weights(
            method=method,
            df=sme_total_growth,
            cols=loo_cols
        )

        loo_index = weighted_index_series(
            df=sme_total_growth,
            weights=loo_weights
        )

        loo_corrs.append(safe_corr(full_index, loo_index))
        loo_msds.append(safe_mean_squared_deviation(full_index, loo_index))

    robustness_summary.append({
        "method": method,
        "S_corr": np.nanmean(loo_corrs),
        "R_min": np.nanmin(loo_corrs),
        "R_max": np.nanmax(loo_corrs),
        "sigma_R": np.nanstd(loo_corrs, ddof=1),
        "S_MSE": np.nanmean(loo_msds),
        "D_min": np.nanmin(loo_msds),
        "D_max": np.nanmax(loo_msds),
        "sigma_D": np.nanstd(loo_msds, ddof=1)
    })

robustness_summary_df = (
    pd.DataFrame(robustness_summary)
    .sort_values("S_corr", ascending=False)
    .reset_index(drop=True)
)

def format_decimal_comma(x):
    return f"{x:.4f}".replace(".", ",")

robustness_table = robustness_summary_df.copy()

robustness_table["method"] = robustness_table["method"].replace({
    "Factor × inverse volatility": "Factor × inverse volatility",
    "Factor analysis": "Factor analysis",
    "PCA × inverse volatility": "PCA × inverse volatility",
    "PCA": "PCA",
    "Inverse volatility": "Inverse volatility",
    "RAW": "RAW"
})

robustness_table = robustness_table.rename(columns={
    "method": "Method",
    "S_corr": "S_corr",
    "R_min": "R_min",
    "R_max": "R_max",
    "sigma_R": "sigma_R",
    "S_MSE": "S_MSE",
    "D_min": "D_min",
    "D_max": "D_max",
    "sigma_D": "sigma_D"
})

robustness_table = robustness_table[
    [
        "Method",
        "S_corr",
        "R_min",
        "R_max",
        "sigma_R",
        "S_MSE",
        "D_min",
        "D_max",
        "sigma_D"
    ]
]

display(
    robustness_table.style
    .format({
        "S_corr": "{:.4f}",
        "R_min": "{:.4f}",
        "R_max": "{:.4f}",
        "sigma_R": "{:.4f}",
        "S_MSE": "{:.4f}",
        "D_min": "{:.4f}",
        "D_max": "{:.4f}",
        "sigma_D": "{:.4f}"
    })
    .hide(axis="index")
)

Output

Method	S_corr	R_min	R_max	sigma_R	S_MSE	D_min	D_max	sigma_D
Factor × inverse volatility	0.9965	0.9937	1.0000	0.0021	0.3822	0.0001	0.8898	0.2878
Factor analysis	0.9960	0.9883	1.0000	0.0036	0.4227	0.0005	1.1238	0.3296
PCA × inverse volatility	0.9959	0.9939	1.0000	0.0019	0.4137	0.0000	0.8143	0.2351
PCA	0.9953	0.9910	1.0000	0.0030	0.4646	0.0000	0.8473	0.2549
Inverse volatility	0.9933	0.9859	0.9969	0.0041	0.5544	0.2250	1.0232	0.2758
RAW	0.9850	0.9254	0.9977	0.0244	1.2057	0.2411	5.6997	1.8345

The leave-one-out robustness results show that all six methods preserve the main time-series pattern of the index, but the hybrid and latent-component-based methods are the most stable. Factor analysis $\times$ inverse volatility, factor analysis, and PCA $\times$ inverse volatility perform similarly well, with very high average correlations and relatively low mean squared deviations. The RAW index remains useful as a simple benchmark, but it is the least robust specification, with the largest sensitivity to the omission of individual indicators.

Final method choice and graphical comparison

Based on the methodological comparison, the final composite SME index is constructed using the PCA × inverse volatility weighting method. The robustness results show that factor analysis × inverse volatility, factor analysis, and PCA × inverse volatility all perform similarly well, but PCA × inverse volatility offers the clearest compromise between statistical robustness, interpretability, and stability. It preserves the PCA logic of capturing common variation across indicators, while the inverse-volatility adjustment reduces the influence of highly volatile components.

The following figures present the selected index from three perspectives. First, the labor, capital, and output subindices are compared with the aggregate SME index. Second, the micro, small, and medium-sized enterprise indices are compared with the SME total. Finally, the SME index is compared with the total economy index using the same weights, so that differences between the two series reflect differences in underlying economic dynamics rather than differences in weighting.

Python codeCell 5

Show code

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

growth_plot = growth.copy()

growth_plot["size_class"] = growth_plot["size_class"].replace({
    "KKV összesen": "SME total",
    "Összesen": "Total economy",
    "0-9": "Micro",
    "10-49": "Small",
    "50-249-": "Medium"
})

labor_cols = [
    "Active enterprises with revenue",
    "Employment, persons",
    "Personnel costs, HUF billion"
]

capital_cols = [
    "Gross investment, HUF billion",
    "R&D, HUF billion"
]

output_cols = [
    "Revenue, HUF billion",
    "Value added, HUF billion",
    "Export revenue, HUF billion"
]

def weighted_average(df, cols, weights):
    cols = [col for col in cols if col in weights.index and col in df.columns]

    if len(cols) == 0:
        return pd.Series(np.nan, index=df.index)

    w = weights[cols].copy()
    w = w / w.sum()

    x = df[cols].astype(float)

    values = []

    for _, row in x.iterrows():
        valid = row.notna()

        if valid.sum() == 0:
            values.append(np.nan)
        else:
            w_valid = w[valid]
            w_valid = w_valid / w_valid.sum()
            values.append(np.sum(row[valid] * w_valid))

    return pd.Series(values, index=df.index)

def weighted_index_row(row, weights):
    available_cols = [col for col in weights.index if col in row.index]
    x = row[available_cols].astype(float)
    valid = x.notna()

    if valid.sum() == 0:
        return np.nan

    w = weights[available_cols][valid]
    w = w / w.sum()

    return np.sum(x[valid] * w)

plot_weights = pca_x_inverse_volatility_weights_final.copy()

sme_total_growth = (
    growth_plot[growth_plot["size_class"] == "SME total"]
    .sort_values("year")
    .copy()
)

sme_total_growth[selected_indicators] = (
    sme_total_growth[selected_indicators]
    .replace([np.inf, -np.inf], np.nan)
)

subindex_pca_iv_plot_df = pd.DataFrame({
    "Year": sme_total_growth["year"].values,
    "Labor": weighted_average(sme_total_growth, labor_cols, plot_weights).values,
    "Capital": weighted_average(sme_total_growth, capital_cols, plot_weights).values,
    "Output": weighted_average(sme_total_growth, output_cols, plot_weights).values,
    "SME total": weighted_average(sme_total_growth, selected_indicators, plot_weights).values
})

sme_style = {
    "marker": "o",
    "linewidth": 3.2,
    "linestyle": "--",
    "color": "black"
}

other_style = {
    "marker": "o",
    "linewidth": 2.2
}

plt.figure(figsize=(11, 6))

plt.plot(
    subindex_pca_iv_plot_df["Year"],
    subindex_pca_iv_plot_df["Labor"],
    label="Labor",
    **other_style
)

plt.plot(
    subindex_pca_iv_plot_df["Year"],
    subindex_pca_iv_plot_df["Capital"],
    label="Capital",
    **other_style
)

plt.plot(
    subindex_pca_iv_plot_df["Year"],
    subindex_pca_iv_plot_df["Output"],
    label="Output",
    **other_style
)

plt.plot(
    subindex_pca_iv_plot_df["Year"],
    subindex_pca_iv_plot_df["SME total"],
    label="SME total",
    **sme_style
)

plt.axhline(0, linestyle="--", linewidth=1)
plt.xlabel("Year")
plt.ylabel("Annual percentage change index")
plt.grid(True, alpha=0.3)
plt.legend(loc="upper left", frameon=True)
plt.tight_layout()
plt.show()

selected_size_classes = [
    "Micro",
    "Small",
    "Medium",
    "SME total"
]

size_index_pca_iv_plot_df = (
    growth_plot[growth_plot["size_class"].isin(selected_size_classes)]
    .sort_values(["size_class", "year"])
    .copy()
)

size_index_pca_iv_plot_df[selected_indicators] = (
    size_index_pca_iv_plot_df[selected_indicators]
    .replace([np.inf, -np.inf], np.nan)
)

size_index_pca_iv_plot_df["PCA × inverse volatility index"] = (
    size_index_pca_iv_plot_df.apply(
        lambda row: weighted_index_row(row, plot_weights),
        axis=1
    )
)

plt.figure(figsize=(11, 6))

for size_class in selected_size_classes:
    g = (
        size_index_pca_iv_plot_df[
            size_index_pca_iv_plot_df["size_class"] == size_class
        ]
        .sort_values("year")
    )

    if size_class == "SME total":
        plt.plot(
            g["year"],
            g["PCA × inverse volatility index"],
            label=size_class,
            **sme_style
        )
    else:
        plt.plot(
            g["year"],
            g["PCA × inverse volatility index"],
            label=size_class,
            **other_style
        )

plt.axhline(0, linestyle="--", linewidth=1)
plt.xlabel("Year")
plt.ylabel("Annual percentage change index")
plt.grid(True, alpha=0.3)
plt.legend(loc="upper left", frameon=True)
plt.tight_layout()
plt.show()

selected_total_classes = [
    "SME total",
    "Total economy"
]

sme_vs_total_pca_iv_df = (
    growth_plot[growth_plot["size_class"].isin(selected_total_classes)]
    .sort_values(["size_class", "year"])
    .copy()
)

sme_vs_total_pca_iv_df[selected_indicators] = (
    sme_vs_total_pca_iv_df[selected_indicators]
    .replace([np.inf, -np.inf], np.nan)
)

sme_vs_total_pca_iv_df["PCA × inverse volatility index"] = (
    sme_vs_total_pca_iv_df.apply(
        lambda row: weighted_index_row(row, plot_weights),
        axis=1
    )
)

plt.figure(figsize=(11, 6))

for size_class in selected_total_classes:
    g = (
        sme_vs_total_pca_iv_df[
            sme_vs_total_pca_iv_df["size_class"] == size_class
        ]
        .sort_values("year")
    )

    if size_class == "SME total":
        plt.plot(
            g["year"],
            g["PCA × inverse volatility index"],
            label=size_class,
            **sme_style
        )
    else:
        plt.plot(
            g["year"],
            g["PCA × inverse volatility index"],
            label=size_class,
            **other_style
        )

plt.axhline(0, linestyle="--", linewidth=1)
plt.xlabel("Year")
plt.ylabel("Annual percentage change index")
plt.grid(True, alpha=0.3)
plt.legend(loc="upper left", frameon=True)
plt.tight_layout()
plt.show()

Output