you're right, I tried that once and I couldn't raise enough points to ask her to swap books with me but in the other hand choosing to meet Holly doesn't block you from any content with Allison.What usually blocks people from Holly is if they choose to get lunch with Alison instead of her in the early game.
That choice blocks the option to swap reviews with her and change a few other scenes with her slightly if you don't make an effort to restore agenda points with her.