Jared says he is “Done” with the group because they wanted to wait until the last minute to tell him about Blue