Cortez started boxing when he was 12 years old alongside his brother Mike. The two continued their amateur careers in the Bronx where Joe Cortez finished with a 43-2 record and a Golden Gloves championship.
He turned professional in 1963 and fought as a bantamweight, going 18-1 before calling it a career.
Cortez said not many fights were available in his weight class and from a money standpoint, for someone starting a family, the $100 purses just weren't enough so he decided out of boxing.
It was at that point, in 1975, that he got into refereeing on the Golden Gloves level. Just six months after his first amateur fight, Cortez moved up to the professional ranks.
Cortez has been the third man in the ring for some of the most historic fights.
On Feb. 20, 1993, Cortez refereed the Julio Cesar Chavez/Greg Haugen fight in Azteca Stadium in Mexico City, an event that holds the world record for the largest attendance at a boxing match at 132,274.
He also refereed George Foreman's victory over Michael Moorer on Nov. 5, 1994, when Foreman became the oldest heavyweight to win a championship at age 45.
What Cortez thinks separates him from his colleagues is his "fair but firm" motto he adopted early in his career.
"I was doing an interview and I was asked how I am during a fight … how I treat the fighters. So I just told them that I was fair but I was firm. And that kind of stuck so I decided to use that during pre-fight instructions," said Cortez.
"Now fighters know me from reputation and they respect me because I've been in this business so long and they know how I am in the ring," Cortez said.
Still, Cortez said there have been times when even he has made mistakes.
"I've made some calls that were judgment calls, like stopping a fight or calling something a knockdown. Then later, I look at the taped replay and realize that I made a mistake," said Cortez. "But in the ring, you don't have the luxury of an instant replay. And with stoppages, you're just looking out for the safety of the fighters and that's where your instinct steps in."
The world-famous Cortez also has made a name for himself outside the ring.
He regularly works with up-and-coming fighters in gyms and helps them use boxing as a means to better their lives.
"There was a fighter back in the '50s named Gasper Ortega who was always helping out in the community and helping the younger guys get into boxing, and he's the one that got me into it," said Cortez. "So when I'm trying to give back to the community, I'm doing for these kids now what he did for me."
In the Imperial Valley, Cortez has given presentations to security staffs regarding gang prevention at Brawley Union High, was a keynote speaker two years ago during Red Ribbon Week for the Imperial County Sheriff's Office and was the keynote speaker at the California Police Activities League awards dinner in Sacramento in 2000, representing the Brawley Police Activities League.
On Saturday, Cortez will return to the Valley as a guest referee for the main event in the sixth annual Battle of the Badges, which pits area law enforcement agents in the squared circle.
Said Brawley PAL director Chris Eaton: "Joe's just unbelievable. He's one of those good, down to earth people that's a really good guy. For him to bring his name and recognition to the Valley when he has no real ties to anything or anyone here is just incredible.
"For someone to go one day from having lunch with the president of the United States and then the next fly to the Valley for something like this just speaks volumes about him. He's just a great guy that will do whatever he can to help kids."