For example, against swinging rook, building Yagura is not good in general. When putting the king into the castle, the square diagonally front of the king becomes a vulnerable point.
In Diag.1, it's gote's turn. Starting to attack with △N-85 ▲S-86 △P-65, it would be better for gote already. The bishop's path is severe for sente. Thus, in case of castling Yagura, attention should be paid whether the opponent's bishop keeps an eye on one's own king indirectly.
This rule is also applied in Yagura vs Rapid Yagura games.
From first move,
▲P-76 △P-84 ▲S-68 △P-34 ▲P-66 △S-62 ▲P-56 △P-54
▲S-48 △S-42 ▲G49-58 △G-32 ▲G-78 △K-41 ▲K-69 △P-74
▲G58-67 △G-52 ▲S-77 △P-64 ▲B-79 △N-73 ▲B-68 △S62-53
▲K-79 △R-62 ▲K-88 (Diag.2)
Gote's form is a kind of rapid attack strategy which omits moves to build Yagura and tries to attack first. In Diag.2, as in Diag.1, with △N-85 ▲S-86 △P-65, it would be better for gote. The king should not be placed on 88 while the opponent's bishop is keeping an eye on it. The attack from △N-85 is one of the weak points of Yagura. Therefore, according to joseki, instead of the three moves, ▲B-68, ▲K-79, ▲K-88 for castling, sente is supposed to make an attacking formation like ▲P-26, ▲P-25, ▲P-36.
At Diag.3, sente's form is called rapid castling, which is one way of building a Yagura castle. When castling with ▲K-88, ▲G-78 after, sente can build Yagura in fewer moves than usual by omitting ▲B-68. But, in this case, attacked with △P-65 ▲Px △Nx, sente would have a little trouble because the king on the 2nd rank is near the opponent's attack, compared with on the 1st rank. Generally, against rapid strategy, sente shouldn't choice the rapid castling. Looking at △S-33 or △P-44 to go for Yagura, sente should do so.
In this way, depending on the opponent's play, a player need to change one's strategy and castle, not just in Yagura games. I think in the opening it's important to keep the balance.