(Original title: Robots have piloted several large-scale exams, exploring open-ended subject scores such as composition)

News reporter Xu Diwei Intern Li Yan

In this year's mid-term examination, Xiangyang City introduced an intelligent online marking system. Video Source Leiyang Radio and Television Station Website (02:51)

For each major exam, marking is a very important part, but it also takes time and effort. With the continuous upgrading of artificial intelligence, robotic marking technology has become more and more mature in recent years.

Recently, relevant personnel of the University of Science and Technology reported to Yu Xin News () that under the organization of the examination center of the Ministry of Education, intelligent marking technology has been used in large-scale examinations in many provinces across the country (such as the entrance examination for senior high school students, adult college entrance examinations, and academic level tests). ) Passed multiple multi-scale pilot verifications.

In the middle school entrance examination in Hubei Province in 2017, Xiangyang City took the lead in introducing the smart assessment system. Liu Chaozhi, dean of the Municipal Education Examination Institute, told the media that “In addition to manual marking, intelligent marking has the advantage of speed Make up for deficiencies in the processing of identical and blank volumes."

Many multi-scale pilot verifications have been conducted on the large-scale test

In March 2016, the Examination Center of the Ministry of Education and the HKUST launched a joint laboratory to jointly conduct research on artificial intelligence technology in intelligent marking, propositions, assessment evaluation and other aspects.

Recently, HKUST News has told Xinhua News that at present, under the organization of the test center, HKUST’s all-disciplinary intelligent marking technology has been tested at the academic level, such as CET 46, as well as college entrance examinations and examinations in many provinces across the country. Many multi-scale pilot verifications have been conducted on large-scale exams such as the adult college entrance examination.

The verification results show that the computer scoring results have reached the level of on-site examination teachers and fully meet the needs of large-scale examinations.

In the past, the analysis of hundreds of thousands and millions of test paper samples required a huge amount of human resources, and the feasibility was very low. However, through accurate image recognition and massive text retrieval technology, it is now possible to quickly check all examination papers and targets. Similar texts, and quickly extract and mark questions that may be problematic.

According to the "Fuyang Evening News" report, unlike the previous year's mid-term examination markings, in 2017, the Fuyang City Entrance Examination Paper in Hubei Province was the first to introduce an intelligent evaluation system. A technician at the reading point said that the smart assessment system can perform workload analysis, list the total amount of each assessment source, and monitor the quality of each teacher's assessment.

Liu Chaozhi, Dean of Fuyang Education Examination Institute, said that with smart data, the score of each question, the average point of the city, which piece of knowledge students mastered well, which piece of education is not in place, can be an education The teaching diagnosis report is more conducive to teacher education and student learning. "Compared with manual marking, intelligent marking has the advantage of not only speeding up the speed of marking, but also making up for the inadequacies in the processing of similar volumes and blank volumes."

According to Gong Xun, a recruiting staff from the Fuyang City Education Examination Institute, the intelligent marking system can cover most of the models. After using the intelligent system, you can search through the massive data and you can accurately determine whether you have copied the model.

On July 19, Liu Chaozhi told the Xinhua News that it will take more time to disclose more information.

HKUST News told Xinhua News that intelligent review uses a deep textural recognition technology based on deep neural network learning, which has reached the level of recognition of Chinese and English handwritten characters. This technology is used in formal examinations, which can assist manual marking, reduce personnel input, reduce the influence of factors such as fatigue and emotion in manual marking, and further improve the efficiency, accuracy, and fairness of manual marking scores, thereby creating a pole for the entire industry. Big change.

In addition, through this technology, the massive and accurate analysis data generated after the entire test taker's test papers were electronically generated also provided powerful materials for subsequent research in teaching and learning, and provided examinations and examinations that could be well applied in the future. The breakthrough of the combination of scoring business. For example, better integration with real classrooms through intelligent scoring and marking.

“There are some technical achievements in the big projects that can be used for the college entrance examination marking, but the fundamental purpose is to introduce artificial intelligence to push the examination paper to the 3.0 era.” In June, the president of HKUST Flywheel Value Wu Xiaoru told the Xinhua News that “The reading 1.0 The era is a pen and paper review. In the 2.0 era, people were organized on the Internet to use machines to automatically review some objective questions. In the era of artificial intelligence, subjective questions can already be automatically reviewed.

It is no longer a dream for the subjective review of machines to be changed

General examinations often include two parts: objective questions and subjective questions. With the answer card and scanner, the objective questions can all be reviewed by the machine. Not only the speed of reading is greatly improved, but also it is more accurate.

Since the 1960s, many foreign experts and scholars have begun to devote themselves to the subjective examination of the machine-reading technology. Various automatic correction systems have emerged. For example, the E-rater system is used in the US MBA and TOEFL exams. . However, most of these systems are aimed at second language compositions, ie non-native language compositions. However, reviewing written compositions in students' native language needs to be judged at a higher level, such as the literary writing of compositions, the linking of texts, and the concept of composition.

In November 2015, HKUST's machine intelligence marking technology was successfully applied in pilot projects in Anqing and Hefei. After analysing the results of the human-computer ratings, the computer has reached or surpassed the artificial scoring level in terms of the score agreement rate, the average score difference, the correlation degree, and the proportion closer to the arbitration score. This means that machine reviews of subjective questions are no longer fanciful.

So, what is the principle and basis for machine review for subjective questions that do not have objective criteria? Wu Xiaoru explained that the essential difference between machine marking and manual marking is the difference in working mechanism. Machines make decisions through statistics, reasoning, and judgment. This is different from people's thinking. In the marking process, the machine adopts intelligent learning. After a group of experts usually review about 500 to 1000 papers, the machine can learn the mode of review of this paper and form a model. This model can form effective treatment and coverage for other papers, and then automatically review other papers according to the model.

For metrics, a group of experts with a high level of literacy is selected first, and the average score given by the group of experts for a group of examination papers is used as a relative standard. After that, compare the final test results of the machine and the results of other testers' tests with the average score of the experts. If the machine and the expert give an average score closer and more relevant, it is considered that the results of the machine review are expected.

"Only a simple or standardized test pattern is actually very easy to cheat, but from the results of many current applications, there is no one way to deceive the machine well," Wu Xiaoru said. "Like Alpha Like Go, Go doesn't mean that you can defeat it by finding an objective and standard routine."

In addition, Wu Xiaoru said that the machine will check out the unique and creative test papers and hand them out manually. In addition, test papers that have made low-level mistakes but new ideas lead to poor test results also need to be judged by on-site testers and experts.

Wu Xiaoru said that in fact, the subjective review of the machine has been verified for a long time. “Many educational experts, front-line teachers, and principals did not agree with the machine grading at the beginning, but through the on-site comparison of the results, these experts finally recognized that the machine is better than the manual test.”

Exploring composition scores automatically

In the research of the subjective machine scoring technology in recent years, the most important thing for the outside world is the language composition scoring technology developed by the HIT-Xinfei Joint Laboratory.

To score an essay, you need to be faced with a very strong view of the text. Which dimensions should the machine judge? How to quantify these dimensions?

According to the researchers, just as the teachers in the country scored with a set of uniform and rigorous standards in the Chinese and college entrance exams, the machines reviewed the composition. The most important thing was for the machine to learn the standard and then read the standard.

That is, teachers first set up a common set of solutions for assessing the quality of a composition, from the levels of handwriting neatness, vocabulary abundance, sentence smoothness, literary acquisition, textual structure, and conception. Afterwards, the machine can use the algorithm to learn from the small amount of artificially scored samples to obtain the composition scoring criteria. For example, there are 2,000 papers in an examination. Starting from the first paper, the machine can learn the teacher's marking method, and when it learns 200 copies, the machine can replace the manual and automatically score the remaining papers intelligently.

In the composition scoring system, the vocabulary richness and conception belong to the content-related features; the wordiness, partial coherence, syntactic correctness and textual structure belong to the expression-related features; In addition, the technology also uses artificial neural networks to deeply express the semantics of the composition, so as to grasp the concept of the article from a macro perspective.

Each of these standards requires sophisticated and sophisticated technology to support it. For example, handwriting recognition technology is needed to determine the degree of handwriting recognition. That is, handwriting characters in a picture are automatically converted into text and the recognition probability is given to indicate the degree of neatness. For another example, to determine whether an essay is off topic, it is necessary to first extract keywords based on the topic content, and expand according to the topics, and at the same time extract the keywords in the essay, and then calculate the similarity between the keyword of the composition and the keyword of the title. In addition, the topic model can also be trained on the large-scale data of the examination to obtain a global topic distribution, and then compared with the topic distribution of the composition to be examined.

The University of Science and Technology of China, participating in the National "863 Program" (National High-Tech Research and Development Program), said that with the development of artificial intelligence technology, in the future, apart from open composition, even political, historical, and geographical subject question machines can Automatically check.

After the automatic machine reading becomes a reality, teachers will have more time and energy to invest in research on creative methods such as teaching methods and teaching methods, so as to provide students with higher quality and more comprehensive education.

electronic accessories for Silicone

Electronic Accessories For Silicone,Silicone Keyboard,Silicone Keyboard Covers,Silicone Keyboard Protector

Nantong Boxin Electronic Technology Co., Ltd. , https://www.ntbosen.com