Avoid Contractor Allowances

We are remodeling an existing house plus 2 new additions. The builder wants a cost plus 20% contract. I trust the builder, but my anxiety comes from paying 20% on upper end materials we're using. For example, if we use the best shingles, why should the builder get 20% for the additional incremental cost of the upper end vs. average shingle? The whole project has these examples because we are essentially using the best of everything. Any thoughts? Thanks.
Tom