Not that which goeth into the mouth defileth a man; but that which cometh out of the mouth, this defileth a man. -Matthew 15:11
Diet should help your religion. It should not be your religion.
Swami Sivananda was providing a teaching from the Hindu Piscean tradition, which can be beneficial. However, garlic, meat, fish, and stimulating, Rajasic foods are fine, but in balance with the other Gunas.
What Swami Sivananda was against was imbalance, and unfortunately much of humanity is in disequilibrium in terms of the food that is consumed. Therefore he gave very radical instructions for teaching genuine yoga practice (raja, bhakti, jnana yoga, etc.).
We do avoid pork and alcohol, since these substances are toxic for the soul. Pork feeds lust, and alcohol can resuscitate dead egos, which is why we abstain from these elements.