Skip to content

Upload WARC from the web UI to index #436

Closed
@ibnesayeed

Description

@ibnesayeed

It will be convenient if admins can run the server and then drag and drop (or use the file selector dialog) to upload WARC files that get indexed automatically. However, this functionality should only be available to admins and should be included only after admin related controls are separated from the landing page as per #213. Uploaded WARCs can also be organized in collections as per #82.

Activity

machawk1

machawk1 commented on Jul 14, 2018

@machawk1
Member

This will create an interesting dynamic between the replay system and indexer. I would be interested in this being implemented without #213 necessarily being a dependency. However, I do agree that the end-goal will be to consider the setup where the replay system and client are not co-located.

Some further tie-in with warcio and decoupling from pywb might help facilitate this functionality as well.

ibnesayeed

ibnesayeed commented on Jul 14, 2018

@ibnesayeed
MemberAuthor

Without isolating and protecting admin functionalities from the public view, the system is only usable by one-person. For example, the publicly accessible test deployment we have, anyone can shut the IPFS daemon down in it and leave others wondering what happened in the middle of their session.

As far as this feature is concerned, I think it will be really very convenient for people to create WARC files using, say, WARCreate and immediately upload it into their IPWB system to see it replayed.

machawk1

machawk1 commented on Jul 14, 2018

@machawk1
Member

@ibnesayeed Do you have ideas as to how this functionality should be displayed within the ipwb replay UI?

ibnesayeed

ibnesayeed commented on Jul 14, 2018

@ibnesayeed
MemberAuthor

Yes, I have some UI ideas, but we can perhaps discus those offline first and draw some diagrams on whiteboard.

machawk1

machawk1 commented on Aug 10, 2018

@machawk1
Member

@ibnesayeed Would you mind describing some of your UI ideas here?

ibnesayeed

ibnesayeed commented on Aug 10, 2018

@ibnesayeed
MemberAuthor

Oh, this was something that we wanted to discuss in person, but could not recall what it was. There are a few ways we can handlel the UI, but I think it is important to first segregate admin functionality from the landing page then we can talk about where to put it.

machawk1

machawk1 commented on Aug 10, 2018

@machawk1
Member

I think hiding the admin UI from client and adding the ability for clients to upload WARCs can be investigated in parallel. I am considering putting the functionality in-place in the replay backend then we can tie it in visually once we have the functional parts ready.

ibnesayeed

ibnesayeed commented on Aug 10, 2018

@ibnesayeed
MemberAuthor

Sure, in that case we can forget about the drag and drop feature for now and simply add a file upload input anywhere to test the backend implementation.

self-assigned this
on Aug 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

    Participants

    @ibnesayeed@machawk1

    Issue actions

      Upload WARC from the web UI to index · Issue #436 · oduwsdl/ipwb