Brahms, the lord of the Undead, resides here. No sunlight shines on this castle of darkness which mysteriously appears with the setting sun and disappears each morning when the sun rises.

Brahms Castle is a dungeon in Valkyrie Profile, situated on an island off the Western coast of Midgard. The interior of the castle is decorated with draperies and paintings, and Brahms' throne room is located on the top floor. Each visit to the Castle is limited by a 4 minute timer. When it runs out, the Castle vanishes, and the party is sent back to the World Map. It is only available in Normal and Hard mode, from Chapter 3 onwards.

Brahms Castle will appear in Chapter 3, but not during a Spiritual Concentration.
Regardless of whether it results in a fight or not, the confrontation with Brahms will drop Lenneth's Seal Level by 10 points, since he suggests that there are things she does not remember.
It is possible to confront Brahms as early as Chapter 3, but if you are aiming for Ending A, it is best to save this confrontation for Chapter 4. It is easier to control Lenneth's Seal Level that way. This does not prevent you from visiting the Castle in Chapter 3.
It is not entirely clear what Brahms urges Lenneth to remember when he first meets her, as she was not involved in the original events of Silmeria's rebellion. It is, however, plausible that she may have learned of them since then, during one of her previous incarnations.
Although it only appears in Valkyrie Profile, Brahms' castle existed long before the events of the game or those of Valkyrie Profile 2: Silmeria, as Falx, one of Silmeria's Einherjar, died while trying to explore it.