BunCho: AI Supported Story Co-Creation via Unsupervised Multitask Learning to Increase Writers’ Creativity in Japanese
DOI: https://doi.org/10.1145/3411763.3450391
CHI '21 Extended Abstracts: CHI Conference on Human Factors in Computing Systems Extended Abstracts, Yokohama, Japan, May 2021
Co-creation with artificial intelligence (AI) is an upcoming trend. However, less attention has been given to the construction of systems for Japanese novelists. In this study, we built “BunCho”, an AI supported story co-creation system in Japanese. BunCho's AI is GPT-2 (an unsupervised multitask language model) trained using a large-scale dataset of Japanese web texts and novels. With BunCho, users can generate titles and synopses from keywords. Furthermore, we propose an interactive story co-creation AI system as a tabletop role-playing game. According to summative studies of writers (N=16) and readers (N=32), 69% writers enjoyed writing synopses with BunCho more than by themselves, and at least one of five common metrics were improved at objective evaluation, including creativity. In addition, 63% writers indicated that BunCho broadened their stories. BunCho showed paths to assist Japanese novelists in creating high-level and creative writing.
ACM Reference Format:
Hiroyuki Osone, Jun-Li Lu, and Yoichi Ochiai. 2021. BunCho: AI Supported Story Co-Creation via Unsupervised Multitask Learning to Increase Writers’ Creativity in Japanese. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI '21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA 10 Pages. https://doi.org/10.1145/3411763.3450391
INTRODUCTION
Advanced development in artificial intelligence (AI) has enabled human users to collaborate with AI in creative activities [7, 8, 12, 13, 14, 16, 20, 23, 24, 30, 35] such as drawing, design, and writing. Recently, with approaching human-level writing performance of language models using unsupervised multitask learning [34] or large-scale models [5], co-creating with AI has become an upcoming trend in writing, particularly for novels and poetry with high-level writing. In this sense, the Japanese novel has a long history with multiple classic novels.
In this study, we are pioneers in challenging the construction of a Japanese novel using co-creation AI and designing appropriate user interfaces for end-users, specifically for Japanese novelists. Substantial work for creative writing support has focused on the improvement of algorithmic performance, whereas few studies have investigated suitable interfaces that can increase diverse and creative writing. For example, recent AI support on creative writing [7, 14] generated the next or entire contents or suggested an outline of a story for given topics or terms. However, these systems might limit the writer's creation on the range of suggested contents due to the AI generation capability, the usage or capabilities of interfaces, or the user's understanding of AI.
In contrast, we propose a co-creation AI for creative novels that can assist in improving users’ written contents, induce writers’ ideas to generate more and diverse creative contents, and even increase the writer's interest in creating his/her own creative writing. Specifically, we build and improve the capabilities of AI interfaces based on interactive feedback from potential users in social networks (i.e., Twitter of Japan), who used BunCho to create their own novel works. In this way, we propose a tabletop role-playing game (TRPG) as an option of co-creation to assist writers in producing more creative and diverse contents, by making the experience of novel creation similar to playing a game.
We have developed BunCho (which means literary style and “sentence bird” in Japanese), a writing web interface that augments the creativity of amateur novelists, with a series of AI tools. First, to make BunCho capable of Japanese language expressions, we trained BunCho's AI by transfer learning with GPT-2 [34], which is one of the proven state-of-the-art language models based on unsupervised multitask learning. We used a large-scale dataset (approximately 40 GB) that comprises collected blog and news articles from multiple Japanese sites and Japanese Wikipedia articles. Further, to build BunCho's AI, which can generate quality creative writing, we collected diverse web novels of multiple categories (more than 110k novels). We also utilized crowd-sourcing by Japanese readers to remove unnecessary expressions from the synopses of collected web novels.
To simplify the co-creation effort with BunCho, we assist users to easily select diverse and creative keywords for automatically generating the related novel titles or synopses, by suggesting keywords based on either templates, topics, or personal social-network contents. Note that we use the method of unsupervised key-phrase extraction [3] for selecting words. To increase writer's interest and idea for creating more creative contents, we further propose an interactive story co-creation based on a game context by our implemented AI TRPG, in which the AI replies and suggests the next story content based on the writer's input context. Our AI TRPG is inspired by AI Dungeon1, which is a text adventure game using AI to generate open-ended story-lines.
We evaluated BunCho through user experiences, the writing ability of AI, and the capabilities of the interfaces as follows. First, in a summative user study of writers, we found that 69% and 63% of writers enjoyed writing with BunCho and considered that BunCho broadened their created stories, respectively. Furthermore, 69% of writers’ synopses co-created with AI were improved in at least one of the five common metrics, including creativity, in a summative user study of thirty-two readers (to evaluate the writer's works), compared with synopses written without BunCho. Finally, we summarized user experiences from common users and professional Japanese novelists and discussed the advantages, potential, and limitations of BunCho to assist in the development of novel co-creation AI for Japanese novelists. We make the following contributions:
- We are pioneers in building a Japanese novel co-creation AI by collecting a large-scale dataset of Japanese texts and web novels and leveraging state-of-the-art language models (GPT-2). We further revised the collected web novels’ synopses by crowdsourcing;
- We designed user interfaces for increasing the writer's creativity and interest to create his/her own creative novels. We proposed interactive story co-creation using an AI TRPG game context and simplified co-creation with AI by suggesting diverse and creative keywords from a customized template, topics, and personal social-network contents;
- We demonstrated the effectiveness of BunCho by showing that more than 50% of the writers enjoyed and considered that BunCho broadened their creative novels. In addition, writers’ synopses were improved by co-creation with AI in at least one of the five common metrics, including creativity. We extensively discussed advantages, potential, and limitations in novel co-creation with AI, particularly for Japanese novelists.
RELATED WORK
Novel and Comic Creation in Japan
Japan has a culture of tolerance and respect for individual artists. One of the differences between the Japanese and American models of anime-related industries is that in Japan, anime directors, manga artists or the works themselves are famous, such as Hayao Miyazaki's ”Spirited Away”, Makoto Shinkai's ”Your Name”, and Eiichiro Oda's ”One Piece”. However, in the United States, although few cartoonists are mentioned, it is often said that Disney and Pixar are famous for their studios. This is symbolized by the fact that in Japan, individuals are celebrated more [32]. Moreover, Japan has events such as Comic Market2 and COMITIA3, fandom and amateur creators distribute creation called ’dojinshi’ in the event; Comic Market 96 in 2019 was announced on its official site as having attracted a total of 730,000 visitors over four days 4. Such creative culture is exported to other countries as ’Cool Japan’. More than 130,000 people attend Comic-Con International5 held in California every July, where comics and related popular art are showcased in widely known as the premiere convention for fans to celebrate. Events featuring Japanese comic culture are also held at ChinaJoy6 in China and the Japan Expo7 in France, where they are very popular [10, 36, 38]. However, in Japan, there is a sense of urgency about Japan's current creation. As critic Hiroki Azuma points out, in post-modern Japan, grand narratives like ideology have lost their appeal and have been replaced by smaller stories that seem to consume the database [2]. Originality is no longer important in this situation. The consumption of stories constructed by ”moe” elements (like character traits) makes it impossible to distinguish between original and copy any longer [6]. Also, there is a sense of urgency regarding the current state of Japanese web fiction. On web novel websites, the novels are supposed to follow a certain pattern to reach the top of the reading rankings or they will not be read. We understand that writers direct considerable effort into creating works that are evaluated on the platform. However, the websites should be the source of more diverse works. In addition, we believe there is a pure joy of creation that does not depend on ranking or being read. Thus, we believe that BunCho can help with this experience, being suitable for those who want to experience a more independent story. Furthermore, in the creation of BunCho, we thought that there was the need to combine the theme that the writers wanted to write about with the randomness and combination provided by AI. We believe that although many people are familiar with the theme they want to write about, they face difficulties when beginning to write. Even if AI could generate novels on its own, it would not be interesting for writers. Therefore, our idea was a function wherein a user could insert the desired theme and the AI would provide a synopsis. There are still some improvements to be applied, such as ensuring consistency of the text and reducing the tendency to overuse certain words in the dataset; however, we believe we have delineated a direction for the future of writing and creation using AI.
Co-Creation by Human-AI
With the increasing capabilities of AI, the use of AI to assist human users in creative activities such as writing, design, drawing, and dance has been studied by many researchers [7, 8, 12, 13, 14, 16, 20, 22, 24, 30, 35]. To effectively cooperate with AI agents, substantial work [7, 11, 13, 28] has focused on how to adapt the behaviors or generated outputs of an AI agent according to the users’ behaviors during executing a task, the feedback given by users, or the task contexts of a broad scope. However, those works focused on the adaptation of AI behaviors while there are still challenging issues in how to correctly use AI based on the understanding of users. Recent studies [16, 21, 29] showed that users might have difficulties in co-working with AI due to non-deterministic output and information overload from AI outcomes. Further, recent advanced AI co-working systems required the users to be familiar with the manipulation or the context of the AI systems, or asked the users to specify the required information before the usage. This increased the effort of utilizing AI co-creation systems, particularly for novices or less-experienced users. Therefore, we design a novel co-creation AI system by referencing the user feedback of potential end-users during the development process. We focus on whether the devised functions or interfaces of BunCho could improve the quality of novel writing co-created with our AI, especially on creativity, or could increase user's interest to create his/her own co-created novels. We therefore provide more options (i.e., from templates, topics, personal data in social networks) to select keywords for generating diverse and creative synopses.
Interactive User Interfaces for Human-AI
Substantial works of AI-powered tools have focused on devising interfaces that can not only be useful for user co-creation but also be coordinated with AI's intelligent modules, such as machine learning, for creative composition [15, 17, 18, 19, 28, 40] such as drawing, design, and music. For example, Peng et al. [31] implemented interfaces to allow writers to interact with computers and achieve consistent goals. Goldfarb-Tarrant et al. [14] solved the shortcomings of traditional human-machine interaction systems in which users were not involved in the story planning process. The model allows users to edit and change previous contents, and control the diversity of the story by adjusting the parameters. Human-computer interaction provides a good perspective for applying story generation. Further, Louie et al. [28] proposed an interface coordinated with an advanced ML model for novice music co-creation. They proposed more semantic or personalized functions for novice users such as directing the AI music generation toward semantic directions (e.g., happier, sadder, more conventional or surprising), and adjusting particular voice lanes to direct the AI generation for personal style music creation. In this study, we therefore focus on devising suitable interfaces and usage situations with our advanced AI of novel co-creation for Japanese amateur novelists. We utilize flexible or diverse interfaces to acquire input keywords. That is, we use different required items or input ways for diverse purposes of novel titles, synopses, phrase suggestions, or AI TRPG. Further, we devise AI TRPG to increase user's interest for generating more creative novel contents or creating his/her own novel works, by making novel co-creating with AI as playing a game with the AI.
Advanced Development in Deep-Learning-based Language Models
Natural language processing (NLP) models or related algorithms are extensively developed in multiple tasks such as question answering [25], machine translation [27], writing summarization [39], or story generation [1, 26, 41], In particular, language models with deep learning such as ELMo [33] (embedding from language models), BERT [9] (bidirectional encoder representation from transformer [37]), and OpenAI GPT-2 [34], are the best practices for innovative results. The success of deep learning in the NLP field has been extended to the natural language generation (NLG) field. Therefore, in this work, we aim to apply the latest NLG technology to the creativity support area.
BUNCHO: AI SUPPORTED STORY CO-CREATION WEB INTERFACES
Japanese-Writer Centered Creative and Diversified Interfaces
We build BunCho, a web site that assists Japanese writers in co-creating novels. First, we focus on reducing co-creation efforts by providing diversified ways for generating creative titles and synopses, which is expected to be useful for novice writers. Further, we assist users in creating more and creative stories by leveraging AI TRPG that users can interactively create stories like a game context.
3.1.1 Reducing Co-Creation Efforts by Providing Diversified and Creative Keywords Inputs Methods for Generating Titles and Synopses. To reduce co-creation efforts, we build an interface assisting users in selecting creative or diverse keywords for generating titles and synopses generation, as shown in Fig. 1. That is, BunCho suggests keywords based on the customized input methods including templates, topics, or personal social-network posts. Especially, we think that randomness and combination are important for creativity. We believe that random generation of keywords could help with creative writing. Therefore, we included a random generation button such that the title keywords in the dataset can be randomly retrieved.
In the interface of generating titles, the genres field is unified with the genre and subgenre of ”Syosetsuka-ni-Narou”8 (which means “Let's become a novelist”). The maximum number of keywords for web novel sites is usually 10, i.e., 10 fields for input were included. We also included a keyword template generation button to set keywords for famous works, which gives an indication of how titles can be generated. The keywords in the template were set by the authors to follow the contents of the work. Further, in the interface of generating synopses, there are 10 new fields for entering synopsis keywords. The synopsis keywords appear more easily in the synopsis if the same word is used multiple times. We included a random generation button such that the synopsis keywords in the dataset can be randomly retrieved. For users who were unable to select synopsis keywords, we also provided an option to obtain keywords from the person's tweets via Twitter.
3.1.2 Co-Creating More Creative Stories by AI TRPG. We assist users in creating more and diverse stories by leveraging AI TRPG. TRPG, also known as a pen-and-paper role-playing game (RPG), “is a form of RPG in which the participants describe their characters’ actions through speech.9”. In AI TRPG of BunCho, users can co-create more story contents by interactive conversation as describing the main characters in the story, and the AI will suggest corresponding options of sentences with the according story background setting and AI TRPG mode. The functions of AI TRPG are shown as Fig. 2. In function (f), users can set the story background with a title, a synopsis, and keywords, or can automatically set the background by templates. Note that users can utilize the previously generated titles, synopses, and keywords for the background setting. In function (g), users can adjust the number of choices and switch between a mode in which the output is conversation only and a mode in which both conversation and ground sentences are output.
We explain insight by using AI TRPG for story co-creation as follows. Some of the options are illogical, but sometimes quite surprising and interesting. As the user progresses through the story, they can create a replay-style story. We prepared several scenario templates for the user to test: Alternate World Reincarnation - The villainous daughter - VRMMO (stands for Virtual Reality Massive Multiplayer Online role-playing game) - Space Momotaro. There is an option for rewriting the synopsis. In this manner, the user can write in different scenarios. It is also possible to use ”Title Generation” and ”Synopsis Generation” to generate a synopsis from keywords and use it for the synopsis of the AI TRPG.
Unsupervised Multitask Learning on Large Amounts of Japanese Novels
There has been considerable discussion regarding to Open AI's GPT models [34], which are modeling of Unsupervised Multitask Learning and have proven to be approaching human-level ability in NLP tasks. We therefore utilize GPT-2 as the modeling of BunCho's AI. To build the co-creation ability of BunCho, we trained GPT-2 with large-scale data of Japanese articles (40 GB) and 110k web novels (110k novels) and were continuously training the GPT models with the duration about one month (around April-May 2020) by using powerful multiple GPU servers (four NVIDIA V100 GPUs).
3.2.1 The Collected Large Japanese Novels and Articles revised by Crowdsourcing. For pre-training, we collated large amounts of text data from Japanese news sites and blog articles. Scrapy10 was used for crawling and Newspaper3k11 was used to extract the body text. We detailed the data collection and data revision by crowdsourcing and the preprocess of data as follows. Because unnecessary expressions were often concentrated at the end, we removed the last eight lines of every article. In addition, data from the Japanese Wikipedia were also used. After processing, approximately 40 GB of text data was collected. We also collected more than 110k web novels from the top rankings of ”Syosetsuka-ni-Narou” and ”Kakuyomu”12. We scraped the data using beautifulsoup413 and linked each genre, keyword, and body text data in the database. We also utilized crowdsourcing by Japanese readers to revise the contents of unnecessary expressions (e.g., book publishing information or rankings in the platforms), which were removed from the synopses of collected web novels.
3.2.2 Unsupervised Multitask Learning with One-Month GPUs Training. To obtain quality AI writing ability, we trained the GPT-2 language model by using multiple GPUs and continued the training process about one-month, including pre-training on Japanese articles and transfer training on Japanese web novels. To explain the writing ability of GPT-2 model, we detailed about (1) the property of GPT-2 language modeling and its learning. Further, we show the text data pre-processing including (2) how we processed tokenization and the selection of tokenizer and (3) the input data to GPT-2 for generating titles, synopses, and sentences.
(1) To obtain a quality GPT-2 language model for synopsis generation conditional on keywords and other factors, we utilized the property of learning language models by autoregressive factorization of the joint distribution of words. We organized the training data of learning language models such that they could generate ground truth synopses based on the information used by the model to generate synopses. Thus, a joint distribution of elements and synopsis was modeled during learning, and a synopsis was generated from the conditional distribution of given elements during inference. (2) In text data pre-processing, we used SentencePiece14 as tokenizer because SentencePiece is considered to be more suitable for tasks such as sentence generation or translation, and it eliminates unknown words in the data and reduces the token size compared to other tokenizers. The number of tokens in SentencePiece was set to 50000, and the tokenizer was trained with more than 60 GB of the collected Japanese articles and novels. (3) We used a special token to indicate the beginning of genres, keywords, a title, a synopsis, and body text and used it at a predicted time to signal the model to begin generating a title or a synopsis. The data were combined with tokens representing categories: title, keywords, title, synopsis keywords, and synopsis. Note that feature words were extracted from the collected synopses using Embedrank [4].
USER STUDY
We conducted a user study of within-subjects design to assess the extent to which BunCho supports user needs and to identify how this affects the user experience of co-creation with AI. Thus, we compared the experience of amateur novelists using BunCho with the experience of writing alone. We asked the following questions to writers:
RQ1: Is BunCho's co-creation with AI more enjoyable than the traditional creative process?
RQ2: Did you become more creative with the use of BunCho?
Methodology
4.1.1 Measures. To investigate the mentioned research questions and evaluate how BunCho could improve the writing quality of the co-created synopsis, we utilized the following five outcome metrics, which are common in writing evaluations [7, 14]. All metrics were rated on a 7-point Likert scale (1=strongly disagree, 7=strongly agree). Each reader was presented with the following questions after reading a synopsis.
Creativity: Is the story of the synopsis creative?
Interestingness: Is the story of the synopsis interesting?
Comprehensibility: Did you understand the synopsis content?
Grammatical correctness: Is the synopsis grammatically correct with a smooth flow?
Consistency of sentences: Is the structure of the synopsis coherent and clear?
4.1.2 Participants of Writers and Readers. We assembled 16 writers and 32 readers from Crowdworks15. To ensure diverse and different levels of writing synopses from common users, we selected amateur writers with varied writing experiences (including novices). To verify the writing experiences of the writers, we required each writer to describe about their writing experiences. Writing experience is shown in Table 1. Furthermore, to obtain objective and diverse rating results from common users, we used a considerable amount of readers (to evaluate the written synopses) and selected readers who are amateur writers. We paid 1,500 yen (US$15) to each writer and 500 yen (US$5) to each reader.
ID | Writing Experience | Novel Works |
---|---|---|
W1 | 0 | 0 |
W2 | 0 | 0 |
W3 | 0 | 0 |
W4 | 3 years | 10 works |
W5 | 1 week | 1 works |
W6 | 5 years | 6 works |
W7 | 1 week | 0 |
W8 | < 1 year | 2 works |
W9 | 0.5 year | 3 works |
W10 | 8 years | 10 works |
W11 | 0 | 0 |
W12 | 1 year | 3 works |
W13 | 0 | 0 |
W14 | 0 | 0 |
W15 | 0 | 0 |
W16 | 0 | 0 |
4.1.3 Procedure. Initially, we asked the writers to work on a synopsis of the science fiction genre, with or without the assistance of BunCho. After completing the work, the writers were asked to self-evaluate their writing experience: whether they enjoyed writing the story by themselves or with the AI, and whether they thought they could expand the scope of the story with the AI compared to writing it by themselves. Subsequently, we asked readers to evaluate the synopses without informing whether it was written with the assistance of BunCho. The evaluation was based on five indicators: creativity, interestingness, comprehensibility, grammatical correctness, and consistency of sentences.
4.1.4 Analysis. As we were interested in investigating how BunCho could assist amateur writers of different levels or writing experience, we compared the writing quality between synopses written with and without co-creation with AI, considering writers with little and good writing experiences. We conducted a paired t-test to assess whether the average values between the two groups (co-creation with AI vs. written without co-creation) were significantly different. We also applied the Bonferroni correction for our five paired comparisons.
Assessment of Novel Co-Creation with BunCho
The evaluation based on the 32 readers’ responses, in Fig. 3, indicated that BunCho approached the writing ability of a writer. In particular, BunCho assisted users in providing creative or interesting writing, as compared with comprehensibility, grammatical correctness, or consistency of sentences.
Furthermore, we evaluated the effect of BunCho on writers with different writing experiences or levels. Fig. 4 (a) and (b), indicates that synopses co-created with AI were worse than synopses written by novice or low-experience writers, on comprehensibility, correctness, or consistency, compared with experienced writers. This might imply that a user needs certain novel-writing experience or writing ability to co-create with BunCho. Furthermore, we observed that the performance with and without BunCho on creativity (or interestingness) was similar. We might infer that BunCho could suggest creative or interesting novel contents or creative ideas for novel creation. In Fig. 5, we detailed the effect of BunCho on each writer. Among them, we found that 11 writers’ (69%) synopses co-created with BunCho were improved in at least one of the five writing metrics compared with those written without BunCho. These eleven writers are W1, W2, W3, W4, W6, W7, W8, W9, W11, W15, and W16.
4.2.1 Writers’ Self-Assessment. In our user study with 16 writers, 69% enjoyed creating synopses with BunCho more than writing by themselves; 63% of writers considered that BunCho broadened the diversity of their co-created stories. Most users were satisfied with the usage experience of co-creating with BunCho and were interested in the novel content generated by BunCho. We might infer that BunCho could bring positive or creative writing experiences in novel co-creation using AI for common novelists.
Writer's Experiences about Co-Creation with BunCho
4.3.1 Creative Story Suggestion and the Assistance for Novice Writers. We summarized the writer's experiences as follows. BunCho could assist in producing creative story contents or providing ideas with interesting phrases for writers who were facing difficulties. As for the positive reactions, the fact that they were able to create a story which they could not think about on their own using the AI and the freshness of the phrasing was mentioned. In addition, for those who have never written a novel before, creating a synopsis was a difficult task, but BunCho made it easier for them (from the writers W1, W2, W4–W9, W11, W12, and W14–W16, where these detailed comments are put in Appendix).
4.3.2 Inconsistent Story Generation on Experienced Writers’ Creation. Nevertheless, unsatisfied comments showed that some writers might not be satisfied with the usage experience, AI generated contents, or inconsistent AI synopses considering their proposed keywords. Negative reactions indicated that they were still not satisfied with the results generated by the AI and that they felt more comfortable writing by themselves. Moreover, for those who had already thought about a synopsis of the story, some were not satisfied with the synopsis generated from keywords because it did not follow the expected storyline. Some also said that the user interface was difficult to understand (from the writers of W2, W3, W5, W6, W8, W10, and W13, where these detailed comments are put in Appendix). Essentially, there are still some improvements to be made to the AI generation results and user interface; however, we have obtained a good response to using AI as an aid for beginners to start creating.
CONCLUSIONS
We propose BunCho, a Japanese novel co-creation AI interface that can generate novel titles and synopses by typing keywords using the Japanese version GPT-2. BunCho assists users in co-creating creative novels in an interactive format similar to TRPG. In our user study, we compared writers on their own written synopsis with those synopsis co-created by BunCho. The results showed that users enjoyed co-creating the synopsis with BunCho more than by themselves. We believe we have delineated one direction for the future of Japanese story writing and co-creating with AI. Furthermore, we plan to evaluate the effect and user experiences of AI TRPG on BunCho's co-creation.
This work was supported by Japan Science and Technology Agency (JST CREST: JPMJCR19F2, Research Representative: Prof. Yoichi Ochiai), and the funding of Center for Artificial Intelligence Research (C-AIR), University of Tsukuba. In addition, we thank Ghelia Inc. for their support of the GPU server for this project.
THE STATISTICAL VALUES OF ASSESSING CO-CREATION WITH BUNCHO
We showed the statistical values of assessing novel co-creation with BunCho as follows. The average values of means in Fig. 3 are from creativity to consistency of sentences (μ1=4.69, μ1=4.73, p=1.0000; μ2=4.10, μ2=4.35, p=0.0819; μ3=4.80, μ3=5.30, p=0.0000003; μ4=5.00, μ4=5.31, p=0.0006; μ5=5.08, μ5=5.38, p=0.0019). The mean values on low-experience writers in Fig. 4 (a) are (μ1=4.66, μ1=4.67, p=1.0000; μ2=4.03, μ2=4.26, p=0.2195; μ3=4.76, μ3=5.31, p=0.0000007; μ4=4.92, μ4=5.31, p=0.0001; μ5=5.02, μ5=5.36, p=0.0019). The mean values on high-experience writers in Fig. 4 (b) are (μ1=4.83, μ1=5.02, p=1.0000; μ2=4.40, μ2=4.70, p=0.7723; μ3=5.00, μ3=5.28, p=0.7326; μ4=5.33, μ4=5.31, p=1.0000; μ5=5.36, μ5=5.49, p=1.0000).
WRITER'S EXPERIENCES ABOUT CO-CREATION WITH BUNCHO
We showed the writer's experiences (relatively positive) in Section 4.3.1 as follows. (W1): ”What I wrote was a reasoning story that could predict the future, but AI was interesting because it depicted a completely unpredictable future”. (W2): ”When I didn't come up with an idea, I felt that it was easy to write a novel as I could get choices from AI”. (W4): ”The synopsis generated completely different from the story I had in mind, and I thought it was interesting in that sense. However, even though the overall storyline was different from my intentions, I feel like I could have been somewhat helpful by tweaking it”. (W5): ”Although it is a synopsis, when I tried to write a novel myself, I had a hard time writing an idea because I had no writing ability. I feel like I've settled down on the usual settings, partly because of the small number of drawers I have in my mind to expand the story. In that respect, AI generation made it easy for me to come up with ideas for development and to write sentences by giving me a start to ideas and creating sentences to some extent”. (W6): ”I thought that the AI assistance wasn't bad because the combination of keywords was more interesting than I expected. However, it's the many generated results, so it's important to have a sense of depression. When I thought that the correct usage was not to win the first time, but to use it only by assisting”. (W7): ”The wording was very fresh”. (W8): ”I was able to work unexpectedly happily”. (W9): ”I had never written a novel in a situation where the theme was set in advance, so I found it a little difficult. There wasn't much reference to AI because it was set incorrectly, but it was interesting to be able to develop the story from an unexpected direction, such as whether there was such a perspective”. (W11): ”When using AI, the initial settings became more interesting than just thinking for myself. However, when I devised a story, I needed knowledge that I didn't have”. (W12): ”It was a very interesting experiment. The endless possibilities of story plots have expanded”. (W14): ”It would have been easier for AI to create the opportunity, but since I had never written a novel, I had a lot of trouble with them”. (W15): ”It was difficult to write a synopsis, but with AI, I knew how to write so that I could see the road that I couldn't see, and I felt that the range of expression had expanded”. (W16): ”I was able to experience ideas that I couldn't think of”.
We showed the writer's experiences (relatively negative) in Section 4.3.2 as follows. (W2): ”However, I think that it will be easier to use if you describe how to use the site and things like HOW TO”. (W3): ”I couldn't even find a hint because it doesn't make sense”. (W5): “However, if I rely solely on AI, there is no doubt that I will not feel that I am writing a novel myself. I find it difficult to adjust how much I should rely on”. (W6): “I also thought that it would take more time to leave the creation of novels to AI. ”. (W8): “However, if I use AI, the fun will be reduced”. (W10): ”Even if the synopsis was generated by AI, there were some parts that did not make sense in Japanese, and it was difficult to make because the story went in a direction that was too different from what I had imagined. The outline of AI is not a story in the first place depending on the keyword, but I would like a comment on how much you can edit it. In order to make a proper synopsis of 300 characters or more, I had to make it different from the one output by AI, but it was difficult to judge experimentally whether it was okay”. (W13): ”It was quite difficult because I had never written a science fiction novel myself”.
- Prithviraj Ammanabrolu, Ethan Tien, Wesley Cheung, Zhaochen Luo, William Ma, Lara J. Martin, and Mark O. Riedl. 2020. Story Realization: Expanding Plot Events into Sentences. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press, 7375–7382. https://aaai.org/ojs/index.php/AAAI/article/view/6232
- Hiroki Azuma. 2009. Otaku: Japan's database animals. U of Minnesota Press.
- Kamil Bennani-Smires, Claudiu Musat, Andreea Hossmann, Michael Baeriswyl, and Martin Jaggi. 2018. Simple Unsupervised Keyphrase Extraction using Sentence Embeddings. In Proceedings of the 22nd Conference on Computational Natural Language Learning, CoNLL 2018, Brussels, Belgium, October 31 - November 1, 2018, Anna Korhonen and Ivan Titov (Eds.). Association for Computational Linguistics, 221–229. https://doi.org/10.18653/v1/k18-1022
- Kamil Bennani-Smires, Claudiu Musat, Martin Jaggi, Andreea Hossmann, and Michael Baeriswyl. 2018. Embedrank: Unsupervised keyphrase extraction using sentence embeddings. arXiv preprint arXiv:1801.04470(2018).
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. CoRR abs/2005.14165(2020). arxiv:2005.14165 https://arxiv.org/abs/2005.14165
- Zhen Troy Chen. 2018. Poetic prosumption of animation, comic, game and novel in a post-socialist China: A case of a popular video-sharing social media Bilibili as heterotopia. Journal of Consumer Culture(2018), 1469540518787574.
- Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A. Smith. 2018. Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories. In Proceedings of the 23rd International Conference on Intelligent User Interfaces, IUI 2018, Tokyo, Japan, March 07-11, 2018, Shlomo Berkovsky, Yoshinori Hijikata, Jun Rekimoto, Margaret M. Burnett, Mark Billinghurst, and Aaron Quigley (Eds.). ACM, 329–340. https://doi.org/10.1145/3172944.3172983
- Nicholas M. Davis, Chih-Pin Hsiao, Kunwar Yashraj Singh, Lisa Li, and Brian Magerko. 2016. Empirically Studying Participatory Sense-Making in Abstract Drawing with a Co-Creative Cognitive Agent. In Proceedings of the 21st International Conference on Intelligent User Interfaces, IUI 2016, Sonoma, CA, USA, March 7-10, 2016, Jeffrey Nichols, Jalal Mahmud, John O'Donovan, Cristina Conati, and Massimo Zancanaro (Eds.). ACM, 196–207. https://doi.org/10.1145/2856767.2856795
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).
- Michael A Elliott. 2018. The globalization of comic-con and the sacralization of popular culture. In Global leisure and the struggle for a better world. Springer, 221–242.
- Judith E. Fan, Monica Dinculescu, and David Ha. 2019. collabdraw: An Environment for Collaborative Sketching with an Artificial Agent. In Proceedings of the 2019 ACM SIGCHI Conference on Creativity and Cognition, C&C 2019, San Diego, CA, USA, June 23-26, 2019, Steven Dow, Mary Lou Maher, Andruid Kerne, and Celine Latulipe (Eds.). ACM, 556–561. https://doi.org/10.1145/3325480.3326578
- Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the Landscape of Creativity Support Tools in HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 04-09, 2019, Stephen A. Brewster, Geraldine Fitzpatrick, Anna L. Cox, and Vassilis Kostakos (Eds.). ACM, 389. https://doi.org/10.1145/3290605.3300619
- Katy Ilonka Gero and Lydia B. Chilton. 2019. Metaphoria: An Algorithmic Companion for Metaphor Creation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 04-09, 2019, Stephen A. Brewster, Geraldine Fitzpatrick, Anna L. Cox, and Vassilis Kostakos (Eds.). ACM, 296. https://doi.org/10.1145/3290605.3300526
- Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng. 2019. Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Demonstrations, Waleed Ammar, Annie Louis, and Nasrin Mostafazadeh (Eds.). Association for Computational Linguistics, 89–97. https://doi.org/10.18653/v1/n19-4016
- James Granger, Mateo Aviles, Joshua Kirby, Austin Griffin, Johnny Yoon, Raniero Lara-Garduno, and Tracy Hammond. 2018. Lumanote: A Real-Time Interactive Music Composition Assistant. In Joint Proceedings of the ACM IUI 2018 Workshops co-located with the 23rd ACM Conference on Intelligent User Interfaces (ACM IUI 2018), Tokyo, Japan, March 11, 2018(CEUR Workshop Proceedings, Vol. 2068), Alan Saidand Takanori Komatsu (Eds.). CEUR-WS.org. http://ceur-ws.org/Vol-2068/milc8.pdf
- Matthew Guzdial, Nicholas Liao, Jonathan Chen, Shao-Yu Chen, Shukan Shah, Vishwa Shah, Joshua Reno, Gillian Smith, and Mark O. Riedl. 2019. Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 04-09, 2019, Stephen A. Brewster, Geraldine Fitzpatrick, Anna L. Cox, and Vassilis Kostakos (Eds.). ACM, 624. https://doi.org/10.1145/3290605.3300854
- David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477(2017).
- Gaëtan Hadjeres, François Pachet, and Frank Nielsen. 2017. DeepBach: a Steerable Model for Bach Chorales Generation. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017(Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 1362–1371. http://proceedings.mlr.press/v70/hadjeres17a.html
- Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong, and Jacob Howcroft. 2019. The Bach Doodle: Approachable music composition with machine learning at scale. CoRR abs/1907.06637(2019). arxiv:1907.06637 http://arxiv.org/abs/1907.06637
- Mikhail Jacob and Brian Magerko. 2015. Interaction-based Authoring for Scalable Co-creative Agents. In Proceedings of the Sixth International Conference on Computational Creativity, Park City, Utah, USA, June 29 - July 2, 2015, Hannu Toivonen, Simon Colton, Michael Cook, and Dan Ventura (Eds.). computationalcreativity.net, 236–243. http://computationalcreativity.net/iccc2015/proceedings/10_3Jacob.pdf
- Pegah Karimi, Mary Lou Maher, Nicholas Davis, and Kazjon Grace. 2019. Deep Learning in a Computational Model for Conceptual Shifts in a Co-Creative Design System. arXiv preprint arXiv:1906.10188(2019).
- Natsumi Kato, Hiroyuki Osone, Kotaro Oomori, Chun Wei Ooi, and Yoichi Ochiai. 2019. Gans-based clothes design: Pattern maker is all you need to design clothing. In Proceedings of the 10th Augmented Human International Conference 2019. 1–7.
- Jina Kim, Soyeon Shin, Kunwoo Bae, Soyoung Oh, Eunil Park, and Angel P. del Pobil. 2020. Can AI be a content generator? Effects of content generators and information delivery methods on the psychology of content consumers. Telematics Informatics 55 (2020), 101452. https://doi.org/10.1016/j.tele.2020.101452
- Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI?: Design Ideation with Cooperative Contextual Bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 04-09, 2019, Stephen A. Brewster, Geraldine Fitzpatrick, Anna L. Cox, and Vassilis Kostakos (Eds.). ACM, 633. https://doi.org/10.1145/3290605.3300863
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=H1eA7AEtvS
- Boyang Li, Stephen Lee-Urban, George Johnston, and Mark Riedl. 2013. Story Generation with Crowdsourced Plot Graphs. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, July 14-18, 2013, Bellevue, Washington, USA, Marie desJardins and Michael L. Littman (Eds.). AAAI Press. http://www.aaai.org/ocs/index.php/AAAI/AAAI13/paper/view/6399
- Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual Denoising Pre-training for Neural Machine Translation. CoRR abs/2001.08210(2020). arxiv:2001.08210 https://arxiv.org/abs/2001.08210
- Ryan Louie, Andy Coenen, Cheng Zhi Huang, Michael Terry, and Carrie J Cai. 2020. Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
- Tiago Machado, Daniel Gopstein, Angela Wang, Oded Nov, Andrew Nealen, and Julian Togelius. 2019. Evaluation of a recommender system for assisting novice game designers. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 15. 167–173.
- Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I Lead, You Help but Only with Enough Details: Understanding User Experience of Co-Creation with Artificial Intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, Montreal, QC, Canada, April 21-26, 2018, Regan L. Mandryk, Mark Hancock, Mark Perry, and Anna L. Cox (Eds.). ACM, 649. https://doi.org/10.1145/3173574.3174223
- Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. 2018. Towards controllable story generation. In Proceedings of the First Workshop on Storytelling. 43–49.
- Sun Peng. 2018. Primary Exploration on the Differentiation of the Animation Industry Models of China, Japan and America in the Context of the Two-dimensional Culture. In 5th International Conference on Education, Language, Art and Inter-cultural Communication (ICELAIC 2018). Atlantis Press.
- Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365(2018).
- Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. (2019).
- Melissa Roemmele and Andrew S. Gordon. 2018. Automated Assistance for Creative Writing with an RNN Language Model. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion, Tokyo, Japan, March 07-11, 2018. ACM, 21:1–21:2. https://doi.org/10.1145/3180308.3180329
- Clothilde Sabre. 2017. French Anime and Manga Fans in Japan: Pop culture tourism, media pilgrimage, imaginary. International Journal of Contents Tourism 1 (2017), 1–19.
- Sandeep Subramanian, Raymond Li, Jonathan Pilault, and Christopher J. Pal. 2019. On Extractive and Abstractive Neural Document Summarization with Transformer Language Models. CoRR abs/1909.03186(2019). arxiv:1909.03186 http://arxiv.org/abs/1909.03186
- Zhihua Wu. 2018. ComicCon-Networked Culture and Participatory Business in the US, Japan, and China. Ph.D. Dissertation. Georgetown University.
- Dongling Xiao, Han Zhang, Yu-Kun Li, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2020. ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, Christian Bessiere (Ed.). ijcai.org, 3997–4003. https://doi.org/10.24963/ijcai.2020/553
- Peng Xu. 2020. Deep learning for free-hand sketch: A survey. arXiv preprint arXiv:2001.02600(2020).
- Lili Yao, Nanyun Peng, Ralph M. Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-Write: Towards Better Automatic Storytelling. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, 7378–7385. https://doi.org/10.1609/aaai.v33i01.33017378
4 https://www.comiket.co.jp/info-a/C96/C96AfterReport.html
7 https://www.japan-expo-paris.com/en/
9 https://en.wikipedia.org/wiki/Tabletop_role-playing_game
11 https://newspaper.readthedocs.io/en/latest/
13 https://pypi.org/project/beautifulsoup4/
14 https://pypi.org/project/sentencepiece/
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
CHI '21 Extended Abstracts, May 08–13, 2021, Yokohama, Japan
© 2021 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8095-9/21/05.
DOI: https://doi.org/10.1145/3411763.3450391