In order to destroy the ice, you will have to use the Scarlet Quartz, this will allow you to improve the attack of your character. The next thing you should do is use a charged attack and interact with the crystal and use the effect to break it.
You must bear in mind that it will take several blows to break it, so you will have to keep trying with the rest of the crystals, which will also allow you to find more waypoints.
Stay warm to avoid freezing with Sheer Cold and keep an eye out for enemies.
That's all you need to know about how to investigate strange ice in Genshin Impact, we hope it was as useful as possible, and you manage to unfreeze all the landmarks that the new location hides.