Maybe... although it's still true that we lost a memory from it, even if you were unsure.

I guess the problem then is if this group lost something after, or have some sort of effect, there's a chance they might not notice until afterwards. The only reason I was able to is because the memory I lost had to do with my wish.