Banished is dying the game itself is coming to a close. More than half the mods are gone and that affects the game. good luck. Check the discussion groups there is a lot of chatter about Banished.

sad times

Make sure you have stockpiles and labourers to move things from the production building to the storage. The Miners/Diggers are notoriously bad at moving things themselves when they finish their work and go eat/sleep. Without labourers your clay might be staying in the pit's storage and not go to a stockpile where it can be used.

You can check the production in the mine/quarry/pit to see how much is made each year, and compare that in the townhall menu to how much is in storage or gets used.

As a side note, production in quarries/mines/extraction buildings of that type tends to be rather slow.