Otdfbt

Short story with a alien planet, government officials must wear exploding medallions

How to prevent "they're falling in love" trope

CAST throwing error when run in stored procedure but not when run as raw query

Am I breaking OOP practice with this architecture?

What method can I use to design a dungeon difficult enough that the PCs can't make it through without killing them?

How to Recreate this in LaTeX? (Unsure What the Notation is Called)

Why didn't Miles's spider sense work before?

Can my sorcerer use a spellbook only to collect spells and scribe scrolls, not cast?

How does a predictive coding aid in lossless compression?

Is it inappropriate for a student to attend their mentor's dissertation defense?

Is there a hemisphere-neutral way of specifying a season?

Can compressed videos be decoded back to their uncompresed original format?

What is a romance in Latin?

What do you call someone who asks many questions?

How can I determine if the org that I'm currently connected to is a scratch org?

Extract rows of a table, that include less than x NULLs

Why is this clock signal connected to a capacitor to gnd?

I would say: "You are another teacher", but she is a woman and I am a man

Why was the shrinking from 8″ made only to 5.25″ and not smaller (4″ or less)?

How to show a landlord what we have in savings?

How dangerous is XSS?

Reverse dictionary where values are lists

Do scales need to be in alphabetical order?

Should I tell management that I intend to leave due to bad software development practices?

Variance of Monte Carlo integration with importance sampling

Monte Carlo Integration for non-square integrable functionsMonte Carlo integration aim for maximum varianceVariance reduction technique in Monte Carlo integrationMonte Carlo integration and varianceMonte Carlo Integration on the Real LineUse Importance Sampling and Monte carlo for estimating a summationSampling / Importance Resampling Poisson WeightsImportance SamplingFind the value of an integral using Monte-Carlo methodOptimal proposal for self-normalized importance sampling

3

3

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

$endgroup$

add a comment | 

3

3

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

$endgroup$

add a comment | 

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

$endgroup$

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

asked 2 days ago

user1799323user1799323

1234

asked 2 days ago

user1799323user1799323

1234

1 Answer
1

active

oldest

votes

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400628%2fvariance-of-monte-carlo-integration-with-importance-sampling%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

answered 2 days ago

Xi'anXi'an

59.1k897365

answered 2 days ago

Xi'anXi'an

59.1k897365