In a saucepan, bring the water to a boil. Remove from heat and add the chai tea bags. Let them steep for 10-15 minutes, depending on your desired strength.
Remove the tea bags and press them against the side of the saucepan to extract any excess liquid. Discard the tea bags.
Return the tea-infused water to low heat, and stir in sweetened condensed milk, almond milk, honey or maple syrup, ground cinnamon, ground ginger, ground cloves, ground cardamom, and ground nutmeg.
Simmer the mixture on low heat for about 15-20 minutes, stirring occasionally.
Remove from heat and let it cool slightly. Stir in vanilla extract.
Strain the mixture through a fine-mesh sieve or cheesecloth to remove any spice particles.
Transfer the chai concentrate to a clean, airtight container and refrigerate.