Machine learning and portfolio selections. II.

Machine learning and portfolio selections. II.

L´aszl´o (Laci) Gy¨orfi¹

1Department of Computer Science and Information Theory Budapest University of Technology and Economics

Budapest, Hungary

September 22, 2007

e-mail: gyorfi@szit.bme.hu www.szit.bme.hu/˜gyorfi www.szit.bme.hu/˜oti/portfolio

Dynamic portfolio selection: general case

x_i = (x_i⁽¹⁾, . . .x_i^(d)) the return vector on day i b=b1 is the portfolio vector for the first day initial capitalS0

S1=S0· hb₁,x1i

for the second day,S₁ new initial capital, the portfolio vector b2=b(x1)

S₂ =S₀· hb₁,x₁i · hb(x₁),x₂i.

nth day a portfolio strategy b_n=b(x₁, . . . ,xn−1) =b(xⁿ⁻¹₁ ) Sn=S0

n

Y

i=1

D

b(xⁱ⁻¹₁ ),x_i E

=S0e^nWn(B) with the average growth rate

Wn(B) = 1 n

n

X

i=1

ln D

b(xⁱ⁻¹₁ ),xi

E .

Dynamic portfolio selection: general case

x_i = (x_i⁽¹⁾, . . .x_i^(d)) the return vector on day i b=b1 is the portfolio vector for the first day initial capitalS0

S1=S0· hb₁,x1i

for the second day,S₁ new initial capital, the portfolio vector b2=b(x1)

S₂ =S₀· hb₁,x₁i · hb(x₁),x₂i.

nth day a portfolio strategy b_n=b(x₁, . . . ,xn−1) =b(xⁿ⁻¹₁ ) Sn=S0

n

Y

i=1

D

b(xⁱ⁻¹₁ ),x_i E

=S0e^nWn(B) with the average growth rate

Wn(B) = 1 n

n

X

i=1

ln D

b(xⁱ⁻¹₁ ),xi

E .

Dynamic portfolio selection: general case

x_i = (x_i⁽¹⁾, . . .x_i^(d)) the return vector on day i b=b1 is the portfolio vector for the first day initial capitalS0

S1=S0· hb₁,x1i

for the second day,S₁ new initial capital, the portfolio vector b2=b(x1)

S₂ =S₀· hb₁,x₁i · hb(x₁),x₂i.

nth day a portfolio strategy b_n=b(x₁, . . . ,xn−1) =b(xⁿ⁻¹₁ ) Sn=S0

n

Y

i=1

D

b(xⁱ⁻¹₁ ),x_i E

=S0e^nWn(B) with the average growth rate

n

log-optimum portfolio

X1,X2, . . . drawn from the vector valued stationary and ergodic process

log-optimum portfolioB^∗ ={b^∗(·)}

E{ln

b^∗(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }

Xⁿ⁻¹₁ =X₁, . . . ,Xn−1

Optimality

Algoet and Cover (1988): IfS_n^∗ =S_n(B^∗) denotes the capital after dayn achieved by a log-optimum portfolio strategyB^∗, then for any portfolio strategyBwith capital S_n=S_n(B) and for any process{X_n}^∞_−∞,

lim sup

n→∞

1

n lnS_n−1 nlnS_n^∗

≤0 almost surely

for stationary ergodic process{X_n}^∞_−∞,

n→∞lim 1

n lnS_n^∗ =W^∗ almost surely, where

W^∗=E

maxb(·) E{ln

b(X⁻¹_−∞),X₀

|X⁻¹_−∞}

is the maximal growth rate of any portfolio.

Optimality

Algoet and Cover (1988): IfS_n^∗ =S_n(B^∗) denotes the capital after dayn achieved by a log-optimum portfolio strategyB^∗, then for any portfolio strategyBwith capital S_n=S_n(B) and for any process{X_n}^∞_−∞,

lim sup

n→∞

1

n lnS_n−1 nlnS_n^∗

≤0 almost surely for stationary ergodic process{X_n}^∞_−∞,

n→∞lim 1

n lnS_n^∗ =W^∗ almost surely, where

W^∗=E

maxb(·) E{ln

b(X⁻¹_−∞),X₀

|X⁻¹_−∞}

is the maximal growth rate of any portfolio.

Martingale difference sequences

for the proof of optimality we use the concept of martingale differences:

Definition

there are two sequences of random variables:

{Z_n} {X_n}

Zn is a function of X1, . . . ,Xn,

E{Z_n|X1, . . . ,Xn−1}= 0 almost surely.

Then{Z_n} is called martingale difference sequence with respect to {X_n}.

A strong law of large numbers

Chow Theorem: If{Z_n}is a martingale difference sequence with respect to{X_n}and

∞

X

n=1

E{Z_n²} n² <∞ then

n→∞lim 1 n

n

X

i=1

Z_i = 0 a.s.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j. E{Z_iZj}

= E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}

= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0

Corollary E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

A weak law of large numbers

Lemma: If{Z_n} is a martingale difference sequence with respect to{X_n}then {Z_n}are uncorrelated.

Proof. Put i <j.

E{Z_iZj} = E{E{Z_iZj |X1, . . . ,Xj−1}}

= E{Z_iE{Z_j |X1, . . . ,Xj−1}}

= E{Z_i ·0}= 0 Corollary

E





 1 n

n

X

i=1

Z_i

!2





= 1

n²

n

X

i=1 n

X

j=1

E{Z_iZ_j}

= 1

n²

n

X

i=1

E{Z_i²}

→ 0

if, for example,E{Z_i²} is a bounded sequence.

Constructing martingale difference sequence

{Y_n} is an arbitrary sequence such that Yn is a function of X1, . . . ,Xn

Put

Zn=Yn−E{Y_n|X1, . . . ,Xn−1} Then{Z_n} is a martingale difference sequence:

Z_n is a function of X₁, . . . ,X_n,

E{Z_n|X1, . . . ,Xn−1}

= E{Y_n−E{Y_n|X1, . . . ,Xn−1} |X1, . . . ,Xn−1}

= 0

almost surely.

Constructing martingale difference sequence

{Y_n} is an arbitrary sequence such that Yn is a function of X1, . . . ,Xn

Put

Zn=Yn−E{Y_n|X1, . . . ,Xn−1} Then{Z_n} is a martingale difference sequence:

Z_n is a function of X₁, . . . ,X_n,

E{Z_n|X1, . . . ,Xn−1}

= E{Y_n−E{Y_n|X1, . . . ,Xn−1} |X1, . . . ,Xn−1}

= 0

almost surely.

Constructing martingale difference sequence

{Y_n} is an arbitrary sequence such that Yn is a function of X1, . . . ,Xn

Put

Zn=Yn−E{Y_n|X1, . . . ,Xn−1} Then{Z_n} is a martingale difference sequence:

Z_n is a function of X₁, . . . ,X_n,

E{Z_n|X1, . . . ,Xn−1}

= E{Y_n−E{Y_n|X1, . . . ,Xn−1} |X1, . . . ,Xn−1}

= 0

almost surely.

Constructing martingale difference sequence

{Y_n} is an arbitrary sequence such that Yn is a function of X1, . . . ,Xn

Put

Zn=Yn−E{Y_n|X1, . . . ,Xn−1} Then{Z_n} is a martingale difference sequence:

Z_n is a function of X₁, . . . ,X_n,

E{Z_n|X1, . . . ,Xn−1}

= E{Y_n−E{Y_n|X1, . . . ,Xn−1} |X1, . . . ,Xn−1}

= 0

almost surely.

Constructing martingale difference sequence

{Y_n} is an arbitrary sequence such that Yn is a function of X1, . . . ,Xn

Put

Zn=Yn−E{Y_n|X1, . . . ,Xn−1} Then{Z_n} is a martingale difference sequence:

Z_n is a function of X₁, . . . ,X_n,

E{Z_n|X1, . . . ,Xn−1}

= E{Y_n−E{Y_n|X1, . . . ,Xn−1} |X1, . . . ,Xn−1}

= 0

Optimality

log-optimum portfolioB^∗ ={b^∗(·)}

E{ln

b^∗(Xⁿ⁻¹₁ ),Xn

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),Xn

|Xⁿ⁻¹₁ }

IfS_n^∗ =S_n(B^∗) denotes the capital after day n achieved by a log-optimum portfolio strategyB^∗, then for any portfolio strategy Bwith capital Sn=Sn(B) and for any process {X_n}^∞_−∞,

lim sup

n→∞

1

n lnSn−1 nlnS_n^∗

≤0 almost surely

Optimality

log-optimum portfolioB^∗ ={b^∗(·)}

E{ln

b^∗(Xⁿ⁻¹₁ ),Xn

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),Xn

|Xⁿ⁻¹₁ }

IfS_n^∗ =S_n(B^∗) denotes the capital after day n achieved by a log-optimum portfolio strategyB^∗, then for any portfolio strategy Bwith capital Sn=Sn(B) and for any process {X_n}^∞_−∞,

lim sup

n→∞

1

n lnSn−1 nlnS_n^∗

≤0 almost surely

Proof of optimality

1

n lnSn = 1 n

n

X

i=1

ln D

b(Xⁱ⁻¹₁ ),Xi

E

= 1

n

n

X

i=1

E{lnD

b(Xⁱ⁻¹₁ ),Xi

E

|Xⁱ₁⁻¹}

+ 1

n

n

X

i=1

lnD

b(Xⁱ⁻¹₁ ),X_iE

−E{lnD

b(Xⁱ⁻¹₁ ),X_iE

|Xⁱ⁻¹₁ }

and 1

n lnS_n^∗ = 1 n

n

X

i=1

E{lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

|Xⁱ⁻¹₁ }

+ 1

n

n

X

i=1

lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

−E{lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

|Xⁱ₁⁻¹}

Proof of optimality

1

n lnSn = 1 n

n

X

i=1

ln D

b(Xⁱ⁻¹₁ ),Xi

E

= 1

n

n

X

i=1

E{lnD

b(Xⁱ⁻¹₁ ),Xi

E

|Xⁱ₁⁻¹}

+ 1

n

n

X

i=1

lnD

b(Xⁱ⁻¹₁ ),X_iE

−E{lnD

b(Xⁱ⁻¹₁ ),X_iE

|Xⁱ⁻¹₁ }

and 1

n lnS_n^∗ = 1 n

n

X

i=1

E{lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

|Xⁱ⁻¹₁ }

+ 1

n

n

X

i=1

lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

−E{lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

|Xⁱ₁⁻¹}

Proof of optimality

1

n lnSn = 1 n

n

X

i=1

ln D

b(Xⁱ⁻¹₁ ),Xi

E

= 1

n

n

X

i=1

E{lnD

b(Xⁱ⁻¹₁ ),Xi

E

|Xⁱ₁⁻¹}

+ 1

n

n

X

i=1

lnD

b(Xⁱ⁻¹₁ ),X_iE

−E{lnD

b(Xⁱ⁻¹₁ ),X_iE

|Xⁱ⁻¹₁ }

and 1

n lnS_n^∗ = 1 n

n

X

i=1

E{lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

|Xⁱ⁻¹₁ }

+ 1

n

n

X

i=1

lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

−E{lnD

b^∗(Xⁱ⁻¹₁ ),X_iE

|Xⁱ₁⁻¹}

Universally consistent portfolio

These limit relations give rise to the following definition:

Definition

An empirical (data driven) portfolio strategyBis called

universally consistent with respect to a class C of stationary and ergodic processes{X_n}^∞_−∞,if for each process in the class,

n→∞lim 1

nlnSn(B) =W^∗ almost surely.

Empirical portfolio selection

E{ln

b^∗(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }

fixed integerk >0 E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } ≈E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} and

b^∗(Xⁿ⁻¹₁ )≈b_k(Xⁿ⁻¹_n−k) =arg max

b(·)

E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} because of stationarity

b_k(x^k₁) = arg max

b(·)

E{lnD

b(x^k₁),X_k+1 E

|X^k₁ =x^k₁}

= arg max

b

E{lnhb,X_k+1i |X^k₁ =x^k₁}, which is the maximization of the regression function

m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Empirical portfolio selection

E{ln

b^∗(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } fixed integerk >0

E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } ≈E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k}

and

b^∗(Xⁿ⁻¹₁ )≈b_k(Xⁿ⁻¹_n−k) =arg max

b(·)

E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} because of stationarity

b_k(x^k₁) = arg max

b(·)

E{lnD

b(x^k₁),X_k+1 E

|X^k₁ =x^k₁}

= arg max

b

E{lnhb,X_k+1i |X^k₁ =x^k₁}, which is the maximization of the regression function

m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Empirical portfolio selection

E{ln

b^∗(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } fixed integerk >0

E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } ≈E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} and

b^∗(Xⁿ⁻¹₁ )≈b_k(Xⁿ⁻¹_n−k) =arg max

b(·)

E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k}

because of stationarity b_k(x^k₁) = arg max

b(·)

E{lnD

b(x^k₁),X_k+1 E

|X^k₁ =x^k₁}

= arg max

b

E{lnhb,X_k+1i |X^k₁ =x^k₁}, which is the maximization of the regression function

m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Empirical portfolio selection

E{ln

b^∗(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } fixed integerk >0

E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } ≈E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} and

b^∗(Xⁿ⁻¹₁ )≈b_k(Xⁿ⁻¹_n−k) =arg max

b(·)

E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} because of stationarity

b_k(x^k₁) = arg max

b(·)

E{lnD

b(x^k₁),X_k+1 E

|X^k₁ =x^k₁}

= arg max

b

E{lnhb,X_k+1i |X^k₁ =x^k₁},

which is the maximization of the regression function m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Empirical portfolio selection

E{ln

b^∗(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ }= max

b(·) E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } fixed integerk >0

E{ln

b(Xⁿ⁻¹₁ ),X_n

|Xⁿ⁻¹₁ } ≈E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} and

b^∗(Xⁿ⁻¹₁ )≈b_k(Xⁿ⁻¹_n−k) =arg max

b(·)

E{ln

b(Xⁿ⁻¹_n−k),X_n

|Xⁿ⁻¹_n−k} because of stationarity

b_k(x^k₁) = arg max

b(·)

E{lnD

b(x^k₁),X_k+1 E

|X^k₁ =x^k₁}

= arg max

b

E{lnhb,X_k+1i |X^k₁ =x^k₁}, which is the maximization of the regression function

m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Regression function

Y real valued X observation vector

Regression function

m(x) =E{Y |X =x} i.i.d. data: Dn={(X₁,Y1), . . . ,(Xn,Yn)} Regression function estimate

m_n(x) =m_n(x,D_n) local averaging estimates

m_n(x) =

n

X

i=1

W_ni(x;X₁, . . . ,X_n)Y_i L. Gy¨orfi, M. Kohler, A. Krzyzak, H. Walk (2002) A Distribution-Free Theory of Nonparametric Regression, Springer-Verlag, New York.

Regression function

Y real valued X observation vector Regression function

m(x) =E{Y |X =x}

i.i.d. data: Dn={(X₁,Y1), . . . ,(Xn,Yn)}

Regression function estimate

m_n(x) =m_n(x,D_n) local averaging estimates

m_n(x) =

n

X

i=1

W_ni(x;X₁, . . . ,X_n)Y_i L. Gy¨orfi, M. Kohler, A. Krzyzak, H. Walk (2002) A Distribution-Free Theory of Nonparametric Regression, Springer-Verlag, New York.

Regression function

Y real valued X observation vector Regression function

m(x) =E{Y |X =x}

i.i.d. data: Dn={(X₁,Y1), . . . ,(Xn,Yn)}

Regression function estimate

m_n(x) =m_n(x,D_n)

local averaging estimates m_n(x) =

n

X

i=1

W_ni(x;X₁, . . . ,X_n)Y_i L. Gy¨orfi, M. Kohler, A. Krzyzak, H. Walk (2002) A Distribution-Free Theory of Nonparametric Regression, Springer-Verlag, New York.

Regression function

Y real valued X observation vector Regression function

m(x) =E{Y |X =x}

i.i.d. data: Dn={(X₁,Y1), . . . ,(Xn,Yn)}

Regression function estimate

m_n(x) =m_n(x,D_n) local averaging estimates

m_n(x) =

n

X

i=1

W_ni(x;X₁, . . . ,X_n)Y_i

L. Gy¨orfi, M. Kohler, A. Krzyzak, H. Walk (2002) A Distribution-Free Theory of Nonparametric Regression, Springer-Verlag, New York.

Regression function

Y real valued X observation vector Regression function

m(x) =E{Y |X =x}

i.i.d. data: Dn={(X₁,Y1), . . . ,(Xn,Yn)}

Regression function estimate

m_n(x) =m_n(x,D_n) local averaging estimates

m_n(x) =

n

X

i=1

W_ni(x;X₁, . . . ,X_n)Y_i L. Gy¨orfi, M. Kohler, A. Krzyzak, H. Walk (2002) A

Correspondence

X ∼ X^k₁

Y ∼ lnhb,X_k+1i

m(x) =E{Y |X =x} ∼ m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Correspondence

X ∼ X^k₁

Y ∼ lnhb,X_k+1i

m(x) =E{Y |X =x} ∼ m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Correspondence

X ∼ X^k₁

Y ∼ lnhb,X_k+1i

m(x) =E{Y |X =x} ∼ m_b(x^k₁) =E{lnhb,X_k+1i |X^k₁ =x^k₁}

Partitioning regression estimate

PartitionP_n={A_n,1,An,2. . .}

A_n(x) is the cell of the partition P_n into which x falls mn(x) =

Pn

i=1Y_iI_[Xi∈An(x)] Pn

i=1I_[Xi∈An(x)]

LetGn be the quantizer corresponding to the partition P_n: G_n(x) =j ifx ∈A_n,j.

the set of matches

I_n(x) ={i ≤n: G_n(x) =G_n(X_i)} Then

mn(x) = P

i∈In(x)Yi

|I_n(x)| .

Partitioning regression estimate

PartitionP_n={A_n,1,An,2. . .}

A_n(x) is the cell of the partition P_n into which x falls

mn(x) = Pn

i=1Y_iI_[Xi∈An(x)] Pn

i=1I_[Xi∈An(x)]

LetGn be the quantizer corresponding to the partition P_n: G_n(x) =j ifx ∈A_n,j.

the set of matches

I_n(x) ={i ≤n: G_n(x) =G_n(X_i)} Then

mn(x) = P

i∈In(x)Yi

|I_n(x)| .

Partitioning regression estimate

PartitionP_n={A_n,1,An,2. . .}

A_n(x) is the cell of the partition P_n into which x falls mn(x) =

Pn

i=1Y_iI_[Xi∈An(x)]

Pn

i=1I_[Xi∈An(x)]

LetGn be the quantizer corresponding to the partition P_n: G_n(x) =j ifx ∈A_n,j.

the set of matches

I_n(x) ={i ≤n: G_n(x) =G_n(X_i)} Then

mn(x) = P

i∈In(x)Yi

|I_n(x)| .

Partitioning regression estimate

PartitionP_n={A_n,1,An,2. . .}

A_n(x) is the cell of the partition P_n into which x falls mn(x) =

Pn

i=1Y_iI_[Xi∈An(x)]

Pn

i=1I_[Xi∈An(x)]

LetGn be the quantizer corresponding to the partition P_n: G_n(x) =j ifx ∈A_n,j.

the set of matches

I_n(x) ={i ≤n: G_n(x) =G_n(X_i)} Then

mn(x) = P

i∈In(x)Yi

|I_n(x)| .

Partitioning regression estimate

PartitionP_n={A_n,1,An,2. . .}

A_n(x) is the cell of the partition P_n into which x falls mn(x) =

Pn

i=1Y_iI_[Xi∈An(x)]

Pn

i=1I_[Xi∈An(x)]

LetGn be the quantizer corresponding to the partition P_n: G_n(x) =j ifx ∈A_n,j.

the set of matches

I_n(x) ={i ≤n: G_n(x) =G_n(X_i)}

Then

mn(x) = P

i∈In(x)Yi

.

Partitioning-based portfolio selection

fixk, `= 1,2, . . .

P_{`</sub>={A<sub>`,j},j = 1,2, . . . ,m_`}finite partitions of R^d,

G_{`</sub> be the corresponding quantizer: G<sub>`}(x) =j, if x∈A<sub>`,j</sub>. G`(xⁿ₁) =G`(x1), . . . ,G`(xn),

the set of matches:

Jn={k <i <n:G_{`</sub>(x<sup>i−1</sup><sub>i−k</sub>) =G<sub>`}(xⁿ⁻¹_n−k)}

b^(k,`)(xⁿ⁻¹₁ ) =arg max

b

X

i∈J_n

lnhb,x_ii

if the setIn is non-void, and b0 = (1/d, . . . ,1/d) otherwise.

Partitioning-based portfolio selection

fixk, `= 1,2, . . .

P_{`</sub>={A<sub>`,j},j = 1,2, . . . ,m_{`</sub>}finite partitions of R<sup>d</sup>, G<sub>`} be the corresponding quantizer: G_{`</sub>(x) =j, if x∈A<sub>`,j}.

G`(x<sup>n</sup><sub>1</sub>) =G`(x1), . . . ,G`(xn), the set of matches:

Jn={k <i <n:G_{`</sub>(x<sup>i−1</sup><sub>i−k</sub>) =G<sub>`}(xⁿ⁻¹_n−k)}

b^(k,`)(xⁿ⁻¹₁ ) =arg max

b

X

i∈J_n

lnhb,x_ii

if the setIn is non-void, and b0 = (1/d, . . . ,1/d) otherwise.

Partitioning-based portfolio selection

fixk, `= 1,2, . . .

P_{`</sub>={A<sub>`,j},j = 1,2, . . . ,m_{`</sub>}finite partitions of R<sup>d</sup>, G<sub>`} be the corresponding quantizer: G_{`</sub>(x) =j, if x∈A<sub>`,j}. G`(x<sup>n</sup><sub>1</sub>) =G`(x1), . . . ,G`(xn),

the set of matches:

Jn={k <i <n:G_{`</sub>(x<sup>i−1</sup><sub>i−k</sub>) =G<sub>`}(xⁿ⁻¹_n−k)}

b^(k,`)(xⁿ⁻¹₁ ) =arg max

b

X

i∈J_n

lnhb,x_ii

if the setIn is non-void, and b0 = (1/d, . . . ,1/d) otherwise.

Partitioning-based portfolio selection

fixk, `= 1,2, . . .

P_{`</sub>={A<sub>`,j},j = 1,2, . . . ,m_{`</sub>}finite partitions of R<sup>d</sup>, G<sub>`} be the corresponding quantizer: G_{`</sub>(x) =j, if x∈A<sub>`,j}. G`(x<sup>n</sup><sub>1</sub>) =G`(x1), . . . ,G`(xn),

the set of matches:

Jn={k <i <n:G_{`</sub>(x<sup>i−1</sup><sub>i−k</sub>) =G<sub>`}(xⁿ⁻¹_n−k)}

b^(k,`)(xⁿ⁻¹₁ ) =arg max

b

X

i∈J_n

lnhb,x_ii

if the setIn is non-void, and b0 = (1/d, . . . ,1/d) otherwise.

Partitioning-based portfolio selection

fixk, `= 1,2, . . .

P_{`</sub>={A<sub>`,j},j = 1,2, . . . ,m_{`</sub>}finite partitions of R<sup>d</sup>, G<sub>`} be the corresponding quantizer: G_{`</sub>(x) =j, if x∈A<sub>`,j}. G`(x<sup>n</sup><sub>1</sub>) =G`(x1), . . . ,G`(xn),

the set of matches:

Jn={k <i <n:G_{`</sub>(x<sup>i−1</sup><sub>i−k</sub>) =G<sub>`}(xⁿ⁻¹_n−k)}

b^(k,`)(xⁿ⁻¹₁ ) =arg max

b

X

i∈J_n

lnhb,x_ii

if the setIn is non-void, and b0 = (1/d, . . . ,1/d) otherwise.

Elementary portfolios

for fixedk, `= 1,2, . . .,

B^{(k,`)</sup> ={b<sup>(k,`)}(·)}, are called elementary portfolios

That is,b<sup>(k,`)</sup><sub>n</sub> quantizes the sequencex<sup>n−1</sup><sub>1</sub> according to the partitionP<sub>`</sub>, and browses through all past appearances of the last seen quantized stringG_`(xⁿ⁻¹_n−k) of lengthk.

Then it designs a fixed portfolio vector according to the returns on the days following the occurence of the string.

Elementary portfolios

for fixedk, `= 1,2, . . .,

B^{(k,`)</sup> ={b<sup>(k,`)}(·)}, are called elementary portfolios

That is,b<sup>(k,`)</sup><sub>n</sub> quantizes the sequencex<sup>n−1</sup><sub>1</sub> according to the partitionP<sub>`</sub>, and browses through all past appearances of the last seen quantized stringG_`(xⁿ⁻¹_n−k) of lengthk.

Then it designs a fixed portfolio vector according to the returns on the days following the occurence of the string.

Combining elementary portfolios

How to choosek, `

small k or small `: large bias

large k and large`: few matching, large variance

Machine learning: combination of experts

N. Cesa-Bianchi and G. Lugosi,Prediction, Learning, and Games. Cambridge University Press, 2006.

Combining elementary portfolios

How to choosek, `

small k or small `: large bias

large k and large`: few matching, large variance Machine learning: combination of experts

N. Cesa-Bianchi and G. Lugosi,Prediction, Learning, and Games.

Cambridge University Press, 2006.

Exponential weighing

combine the elementary portfolio strategiesB^{(k,`)</sup>={b<sup>(k,`)}_n }

let{q_{k,`</sub>} be a probability distribution on the set of all pairs (k, `) such that for allk, `,q<sub>k,`}>0.

forη >0 put

wn,k,`=qk,`e^{ηlnSn−1(B(k,`))} forη = 1,

w_{n,k,`</sub>=q<sub>k,`}e<sup>lnSn−1(B(k,`))</sup>=q<sub>k,`</sub>Sn−1(B^(k,`)) and

v<sub>n,k,`</sub>= wn,k,`

P

i,jw_n,i,j. the combined portfoliob:

b_n(xⁿ⁻¹₁ ) =X

k,`

v<sub>n,k,`</sub>b<sup>(k,`)</sup>_n (xⁿ⁻¹₁ ).

Exponential weighing

combine the elementary portfolio strategiesB^{(k,`)</sup>={b<sup>(k,`)}_n } let{q_{k,`</sub>} be a probability distribution on the set of all pairs (k, `) such that for allk, `,q<sub>k,`}>0.

forη >0 put

wn,k,`=qk,`e^{ηlnSn−1(B(k,`))} forη = 1,

w_{n,k,`</sub>=q<sub>k,`}e<sup>lnSn−1(B(k,`))</sup>=q<sub>k,`</sub>Sn−1(B^(k,`)) and

v<sub>n,k,`</sub>= wn,k,`

P

i,jw_n,i,j. the combined portfoliob:

b_n(xⁿ⁻¹₁ ) =X

k,`

v<sub>n,k,`</sub>b<sup>(k,`)</sup>_n (xⁿ⁻¹₁ ).

Exponential weighing

combine the elementary portfolio strategiesB^{(k,`)</sup>={b<sup>(k,`)}_n } let{q_{k,`</sub>} be a probability distribution on the set of all pairs (k, `) such that for allk, `,q<sub>k,`}>0.

forη >0 put

wn,k,`=qk,`e^{ηlnSn−1(B(k,`))}

forη = 1,

w_{n,k,`</sub>=q<sub>k,`}e<sup>lnSn−1(B(k,`))</sup>=q<sub>k,`</sub>Sn−1(B^(k,`)) and

v<sub>n,k,`</sub>= wn,k,`

P

i,jw_n,i,j. the combined portfoliob:

b_n(xⁿ⁻¹₁ ) =X

k,`

v<sub>n,k,`</sub>b<sup>(k,`)</sup>_n (xⁿ⁻¹₁ ).

Exponential weighing

combine the elementary portfolio strategiesB^{(k,`)</sup>={b<sup>(k,`)}_n } let{q_{k,`</sub>} be a probability distribution on the set of all pairs (k, `) such that for allk, `,q<sub>k,`}>0.

forη >0 put

wn,k,`=qk,`e^{ηlnSn−1(B(k,`))} forη = 1,

w_{n,k,`</sub>=q<sub>k,`}e<sup>lnSn−1(B(k,`))</sup>=q<sub>k,`</sub>Sn−1(B^(k,`))

and

v<sub>n,k,`</sub>= wn,k,`

P

i,jw_n,i,j. the combined portfoliob:

b_n(xⁿ⁻¹₁ ) =X

k,`

v<sub>n,k,`</sub>b<sup>(k,`)</sup>_n (xⁿ⁻¹₁ ).

Exponential weighing

combine the elementary portfolio strategiesB^{(k,`)</sup>={b<sup>(k,`)}_n } let{q_{k,`</sub>} be a probability distribution on the set of all pairs (k, `) such that for allk, `,q<sub>k,`}>0.

forη >0 put

wn,k,`=qk,`e^{ηlnSn−1(B(k,`))} forη = 1,

w_{n,k,`</sub>=q<sub>k,`}e<sup>lnSn−1(B(k,`))</sup>=q<sub>k,`</sub>Sn−1(B^(k,`)) and

v<sub>n,k,`</sub>= wn,k,`

P

i,jw_n,i,j.

the combined portfoliob: b_n(xⁿ⁻¹₁ ) =X

k,`

v<sub>n,k,`</sub>b<sup>(k,`)</sup>_n (xⁿ⁻¹₁ ).