Polls say Abbott is own worst enemy