Prior to 1942, Stockton and greater San Joaquin County were home to one of the largest Japanese communities in the United States. The community was deeply engaged in local industries, published its own newspapers, and was centered on the historic Nihonmachi, or Japanese Quarter, in downtown Stockton. After Executive Order 9066 was signed by President Franklin D. Roosevelt, the US Department of War requisitioned the San Joaquin County Fairgrounds to be used as an incarceration site for the Japanese community. On May 10, 1942, thousands of local Japanese Americans were forcibly incarcerated at the Assembly Center. Inmates were forced to live in the Fairgrounds horse-barns and were forbidden from leaving the camp. The Assembly Center operated until October 17, 1942, at which point nearly the entire population was deported to Rohwer, Arkansas. The cultural, social, and spiritual life of the Assembly Center has been richly documented by organizations such as Densho, and several oral histories of camp survivors have been conducted by Sacramento State University, the University of the Pacific, and the San Joaquin County Historical Museum.

After the war, the US Department of War and the San Joaquin County Fairgrounds demolished or auctioned off many of the structures associated with the Assembly Center. In 1980, local civic leader Richard Yoshikawa partnered with the Stockton Cultural Heritage Board to have the site designated a State Historic Landmark (#934), and were able to have a Historic Marker erected at the site. By that point, the only building at the site original to the Assembly Center was the Assembly Center Hospital Building, built in 1939 as a County Museum building. In 2021, California Governor Gavin Newsom issued an executive order listing the San Joaquin County Fairgrounds as excess state property to be redeveloped into housing. The Historical Museum communicated with local leaders in the Japanese American community and led a letter-writing campaign to the Governor’s Office and the State Department of General Services to preserve the building. The letter campaign was ultimately successful

In May of 2024, museum staff attended the Japanese American Memorial Pilgrimage to Rohwer, Arkansas to present on the status of the building and efforts to preserve the site. At the pilgrimage, several community leaders were inspired to organize toward developing a long-term vision for the building that would memorialize the site and provide interpretation for future generations. In January of 2025, those leaders formed an ad hoc committee of the museum, Okagesama (translated as “thanks to you”), to spearhead efforts to establish an Interpretive Center and Memorial Site.