Jazzy Jay was born in South Carolina. He moved with his family to New York City at a young age and took up the drums, his first instrument. He began his career in hip hop in the 1970s in The Bronx, the heart of hip hop culture during its early development. An early member of Afrika Bambaataa's Universal Zulu Nation, he was a protegé of Bambaataa as well as his cousin, Kool DJ Red Alert. Beginning as a Zulu King dancer in the early 1970s, Jay later became a Universal Zulu Nation DJ and was a member of the Zulu group Jazzy Five, with which he recorded the single "Jazzy Sensation."

Although Jazzy Jay began performing primarily at street parties, in the 1980s he began DJ’ing in New York clubs such as Negril, the Roxy, the Ritz, and Danceteria. He also hosted a hip hop radio program on KISS-FM and in 1984 he played the part of a club DJ in the influential hip hop film Beat Street. Jazzy Jay and Afrika Islam performed some of the first DJ team routines, and teamed up to battle both Grandmaster Flash (solo) and Grand Wizard Theodore (solo).

Around 1984, Jay met Rick Rubin and assisted him in laying the foundation for what would become Def Jam Recordings. The label's first official single was the single "It's Yours" by T La Rock and Jazzy Jay. Jay later introduced Rubin to Russell Simmons, creating one of the most important partnerships in hip hop production. Jazzy Jay also put out Def Jam's third 12" in 1985, entitled "Def Jam" b/w "Cold Chillin' In The Spot," which featured Russell Simmons on vocals. In 1986, he participated in the recording Planet Rock - The Album, which went gold. Also a producer, Jay founded Jazzy Jay's Studio in the Bronx, where he produced early recordings by Diamond D, Fat Joe, Brand Nubian, A Tribe Called Quest, and others. He later began his own label, Strong City Records, through a partnership with Rocky Bucano.