The citrus definitely got named first. The earliest recorded use of orange the fruit in English is from the 1300's and came to us from the old french orenge adapted from the Arabic naranj, from the Persian narang, from the Sanskrit naranga meaning (orange tree).
Hope this helped