How much zinc to take for boosting testosterone?

As you know I have discussed previously the critical role of zinc in raising testosterone and libido in middle aged men.  As a (for all practical purposes) vegetarian (though, once I recognized that I must do something about diet, I have been consuming animals occasionally), I must have been definitely deficient in zinc, since vegetarian dishes simply do not have enough zinc, and if you are already experiencing a rapid decline in Testosterone, you definitely need raising your zinc consumption.

Now, if you are a meat eater, particularly if you can consume oysters more frequently, there is a lot less to worry, but for me, I need to take supplements.  But like anything else, you need to be careful how much you take.  For instance, the multivitamin tablet that I take already has 15 mg of zinc in it.  I also occasionally take a scoop of Trade Joe's Whey Protein and that too has added zinc in it.  So the zinc supplement that I bought needs to be taken carefully.  It has recommended two pills daily and that would give me 30 mg zinc, but according to recommendations, a healthy male should be taking about 10-15 mg zinc daily.  So I am being careful and taking only one pill of the zinc right now and maybe the day I consume the whey protein I will not take the pill.  Or the days I realize that my diet is not very rich, I might pop two pills.  Doctors suggest that the daily consumption should not exceed 40 mg and that too only for short periods of time while you raise your Testosterone level.  My hope is that if I get my original libido back or even if it improves significantly, I might stop the pills and just get my zinc from the multi-vitamin pill.