Also, treatment for a brain tumor, such as surgery or radiation therapy, can affect the pituitary gland and cause hypogonadism. Inflammatory disease. Certain inflammatory diseases, such as sarcoidosis, histiocytosis and tuberculosis, involve the hypothalamus and pituitary gland and can affect testosterone production. HIV/AIDS. HIV/AIDS can cause low levels of testosterone by affecting the hypothalamus, the pituitary and the testes. Medications. The use of certain drugs, such as opiate pain medications and some hormones, can affect testosterone production. Obesity. Being significantly overweight at any age might be linked to hypogonadism. Aging. As men age, there's a slow, progressive decrease in testosterone production. The rate varies greatly.