From A Wiki of Ice and Fire
The following pieces of information can be seen as evidence:
- Olyvar is related to Gyles Rosby. Olyvar's mother Bethany was born a Rosby. As some of Olyvar's relatives have been fostered at their maternal family members (e.g., Merrett Frey served as page and squire to Lord Sumner Crakehall, related to his mother Amarei Crakehall; Geremy Frey sent his son and daughter by Carolei Waynwood, to Ironoaks, the seat of his wife's relatives of House Waynwood, as squire and ward), it is possible that one of Bethany's children was likewise fostered at her relative's seat.
- In that case, Olyvar appears the most likely option, as his older brothers Perwyn and Benfrey had already been knighted by A Game of Thrones, and thus are unlikely to have still been a ward at a lord's castle. In addition, Benfrey is a married man with children, while Perwyn is present at the Siege of Riverrun in 300 AC. In addition, Benfrey died at the Red Wedding, while the ward was mentioned afterwards, eliminating him as a possibility. Willamen is a sworn maester in service at Longbow Hall in the Vale of Arryn, while Roslin resides at the Twins. Out of all of Bethany's children, this leaves only Olyvar as the last remaining possibility.
- The maester serving at Rosby, Gyles Rosby's seat, is Melwys Rivers, a bastard son of Lord Walder Frey.
- The ward would not allow Lady Falyse Stokeworth and her husband, Ser Balman Byrch to Rosby. As a loyalist to Robb Stark, Olyvar would have plenty of reasons not to allow a Lannister loyalist access to a seat he controlled.
- Born in 281 AC, Olyvar would have reached the age of majority, and thus be able to make decisions without the approval of a castellan.
- Falyse Stokeworth calls Rosby's ward an "ill-born wretch", possibly suggesting low birth, and not being a member of a noble house like House Frey.
- Cersei Lannister claims that the ward is "not of [Rosby's] blood", suggesting that Rosby and the ward are not related.
- "Not of [Rosby's] blood" could indicate that the ward does not descend from Rosby, although he could still be related to Rosby in other ways.
The following links are a selection of the many places online where fans have discussed or outlined the theory and given detailed argumentation:
- Wars and Politics of Ice and Fire: Heirs in the Shadows: The Ward at Rosby.