Consistent Sample from GP

[HanyangHenry-Wang]

Hi all,

I am now trying to write a GP-generated objective function but I found that "rsample" and "sample" will not give the consistent GP sample. To be more specific, "rsample" and "sample" will give different value at the same X, which means that the underlying GP samples are different.

What I would like to do is to sample X repeatedly from the same GP sample. Is it possible to do it with gpytorch? Thank you!

Here is my code and the result:

seed = 42
torch.manual_seed(seed)
if torch.cuda.is_available():
    torch.cuda.manual_seed(seed)

with torch.no_grad():
    test_x = torch.linspace(0, 1, 1)  
    test_x2 = torch.linspace(0, 1, 1)

    observed_pred = likelihood(model(test_x))
    samples = observed_pred.sample(torch.Size([1]))

    observed_pred2 = likelihood(model(test_x2))
    samples2 = observed_pred2.sample(torch.Size([1]))  

# Print the samples
print("X are ",test_x)
print('sample value are: ', samples)
print("X are ",test_x2)
print('sample value are: ', samples2)



X are  tensor([0.])
sample value are:  tensor([[0.7332]])
X are  tensor([0.])
sample value are:  tensor([[0.5413]])

[gpleiss]

Is this unexpected? The samples are random, and so calling sample or rsample multiple times should return different answers.

[HanyangHenry-Wang]

Thank you for your reply.

[jacobrgardner]

rsample on GPyTorch's MultivariateNormal can take a set of base samples (the standard Normal samples that are transformed to the correct distribution):

gpytorch/gpytorch/distributions/multivariate_normal.py

Line 198 in 981edd8

def rsample(self, sample_shape: torch.Size = torch.Size(), base_samples: Optional[Tensor] = None) -> Tensor:

If you specify base samples as a fixed torch.randn of the correct shape, you'll get the same sample each time.

[HanyangHenry-Wang]

Thank you for your kind reply. I will have a try.