We had $329.4 million of borrowings at December 31, 2020, compared to $296.9 million of borrowings at December 31, 2019. Our borrowings consisted solely of Federal Home Loan Bank of New York advances. Of that total, $109.0 million of the borrowings are associated with longer-dated swap agreements.
Shareholders’ total equity decreased by $32.0 million, or 13.5%, to $205.6 million at December 31, 2020 compared to $237.6 million at December 31, 2019. The decreased was due primarily to a net loss of $31.5 million for the year ended December 31, 2020.
Comparison of Operating Results for the Years Ended December 31, 2020 and 2019
General. Net income decreased $37.0 million, to a net loss of $31.5 million for the year ended December 31, 2020, compared to net income of $5.5 million for the year ended December 31, 2019. The decrease was due primarily to a $14.2 million valuation allowance as certain Bank property was reclassified to held for sale, goodwill impairment of $15.5 million, a $3.9 million decrease in net interest income and provisions for loan losses of $2.5 million.
Interest Income. Interest income decreased $3.2 million, or 4.9%, to $61.6 million for the year ended December 31, 2020 from $64.8 million for the year ended December 31, 2019. The decrease was due to a decrease of $4.5 million, or 7.7%, in interest income from loans as the average balance of loans decreased $62.7 million to $1.39 billion and the average yield on loans decreased 14 basis points to 3.90% for 2020 from 4.04% for 2019. These were partially offset by increases in the average balances of mortgage-backed securities and other interest-earning assets of $67,000 and $127,000, respectively.
Interest Expense. Interest expense increased $651,000, or 3.0%, to $22.6 million for the year ended December 31, 2020 compared to $21.9 million for the year ended December 31, 2019, due to an increase of $1.6 million in interest expense on borrowings, partially offset by a decrease of $905,000 in interest expense on deposits. Average deposit balances rose to $1.33 billion at a weighted average interest rate of 1.20% during 2020 compared to $1.26 billion at 1.33% during 2019.
Net Interest Income. Net interest income decreased $3.9 million, or 9.0%, to $39.1 million for the year ended December 31, 2020 from $42.9 million for the year ended December 31, 2019, as a result of an increase in interest expense and a decrease in interest income.
Provision for Loan Losses. Provisions for loan losses are charged to operations to establish an allowance for loan losses at a level necessary to absorb probable and incurred losses inherent in our loan portfolio that are both probable and reasonably estimable at the date of the consolidated financial statements. In evaluating the level of the allowance for loan losses, management analyzes several qualitative loan portfolio risk factors including, but not limited to, management’s ongoing review and grading of loans, facts and issues related to specific loans, historical loan loss and delinquency experience, trends in past due and non-accrual loans, existing risk characteristics of specific loans or loan pools, the fair value of underlying collateral, current economic conditions and other qualitative and quantitative factors which could affect potential credit losses.
After an evaluation of these factors, particularly in light of the COVID-19 pandemic and the increase in our special mention and classified loans from $5.5 million at December 31, 2019 to $31.0 million at December 31, 2020, we recorded a provision for loan losses of $2.5 million for 2020 compared to a provision of $1.3 million for the year ended December 31, 2019. Our allowance for loan losses and letters of credit and commitments was $17.0 million at December 31, 2020 compared to $14.5 million at December 31, 2019. The allowances to total loans was 1.34% at December 31, 2020 compared to 1.03% at December 31, 2019, while the allowance for loan losses to non-performing loans was 131.9% at December 31, 2020 compared to 295.4% at December 31, 2019. We had charge-offs of $59,000 and no recoveries during the year ended December 31, 2020 compared to $15,000 of charge-offs and $25,000 of recoveries during the year ended December 31, 2019.
Non-interest Income. Non-interest income decreased $1.4 million, or 54.0%, to $1.2 million for the year ended December 31, 2020 from non-interest income of $2.6 million for the year ended December 31, 2019. The primary drivers of this decrease were the valuation allowance taken on the real estate owned component of the headquarters complex, which amounted to $1.2 million, and a decrease in fees and service charges of $324,000.
Non-interest Expense. Non-interest expense increased $40.2 million, or 108.7%, to $77.1 million for the year ended December 31, 2020 from $37.0 million for the year ended December 31, 2019. The drivers of this increase included $15.5 million of goodwill impairment, $12.8 million from the reclassification of certain headquarters property to held-for-sale, and increased information technology expenses of $5.4 million.
52