Drive me/you up a wall, To

1. A person or thing that angers, irritates or frustrates.