Sitting behind his desk in his Brawley office, his law library surrounding him and his law degree on the wall beside him, Schmitt reflected on the topic that has been the guiding source for his work.
"The law is a very vital aspect of life today," he said. "The law teaches you how to research and how to cope with problems and challenges."
For 25 years Schmitt was the U.S. magistrate in Imperial County, presiding over federal cases, dealing with issues such as illegal immigration and the movement of drugs across the border.
He was appointed to the position in 1975. It was a part-time position, to which he would be appointed six more times by judges in the federal Southern District Court of California in San Diego.
Officially, Schmitt ended his last four-year term in October 1999, but his tenure was extended by six months when the person who was to replace him opted not to take the job. His tenure was extended again through what is called recall service when it was determined the local magistrate position would become full-time.
Because it took time to work out the logistics of making the position full-time, Schmitt continued as magistrate until local Superior Court Judge Roger Benitez was appointed.
Those who worked with Schmitt said he was a fair judge who treated people with respect.
Louisa Porter, presiding magistrate for the Southern District, said she was not surprised to hear Schmitt was reluctant to be interviewed. She said he was a dependable judge who served "in his quiet and capable way."
She added, "He is respectful and respected. He is a gentleman of the old school. When you think of what a judge should look like, sound like, be like — it is Joe."
Respect is an important word for Schmitt.
"I always made an attempt to be respectful of everyone who came into the court, particularly including the defendants," he said. "The judge is really in a unique position. When you treat people with respect, most times they will return that respect."
Schmitt's life began in New Albany, Ind., in 1923. He attended New Albany High School. Schmitt joked that while he lived in an area where basketball was most popular, he was no great player. He tried out for his high school team but didn't quite make the cut.
After graduating from high school in 1941 he went to Panama to be with family and "to have an adventure away from Indiana."
Moving to Panama turned out to be a major move in his life. It was there that he met the woman who would become his wife. He and his wife, Jean, have been married for 56 years.
The Schmitts were married after they returned to the States. Shortly after their marriage, Joseph, who had been drafted into the Army, went off to World War II.
After training, Schmitt was commissioned as a second lieutenant and was slated to take part in the invasion of Japan. The invasion never occurred because Japan surrendered after the United States dropped two atomic bombs on the country.
In 1946 Schmitt left the Army and attended Indiana University. He graduated in 1948 with a bachelor's degree in finance, then earned a law degree from Stanford University in 1951.
Schmitt pointed out that one of his fellow students at Stanford was William Rehnquist, now chief justice on the U.S. Supreme Court. Another fellow student was Sandra Day O'Connor, a justice on the high court.
In 1951 Schmitt was recalled into the Army for the Korean War and was assigned to the judge advocate general corps. Schmitt was discharged in 1953 and he and Jean moved to the Imperial Valley.
They came to the Valley in a convertible with the top down. The weather was warm and the Schmitts knew this was where they wanted to make their permanent home, according to Jean.
Schmitt first worked in the D.A.'s Office, during which time he was an adviser to the county Board of Supervisors.
He later started his own practice, and was hired as a part-time city attorney for the city of Brawley, a position he held from 1961 to 1991.
In 1975 he decided to throw his hat into the ring when the part-time federal magistrate position became available. He said he never expected to be appointed, but that is exactly what happened.
Marilyn Huff, chief judge of the Southern District, said Schmitt has been an important part of the court system.