Failed to login. Please check the content of your cookies.txt
I used the recommended Firefox extension to download the cookies. Once yesterday and again today. I’ve confirmed that it’s reading the file by renaming it to something wrong and getting a “not found” error. I downloaded all the cookies, not just google ones. I logged out and back in and downloaded the cookies again.
I can see that I’m a manager because I have the “manage group” options.
I’ve triple-rechecked that I’m using the right group name by copy-pasting and seeing that it’s a group name format and not a domain name one.
Is something broken or is it just me?
@gerhard, sorry for the call-out, but have you a quick suggestion on how to debug this? Maybe a login endpoint has changed?
EDIT: Found it. I’ll submit a PR shortly. The endpoint for login changed and I managed to guess the new one. 
https://github.com/discourse/discourse/pull/9432
Newbie trying to import mbox files from Yahoo groups. I’ve followed these instructions several times but always with the same error message. I see others have been successful so it is likely a newbie mistake. The error appears to indicate that split_regex: "^From .+@.+" is not finding the email key to split the file but I tested the regex in a text editor and it works as expected. Line 2 of the import file is similar to Message-ID: <[email protected]>
Any ideas? TIA…
The mbox import is starting...
Traceback (most recent call last):
12: from script/import_scripts/mbox.rb:9:in `<main>'
11: from script/import_scripts/mbox.rb:10:in `<module:ImportScripts>'
10: from script/import_scripts/mbox.rb:12:in `<module:Mbox>'
9: from script/import_scripts/mbox.rb:12:in `new'
8: from /var/www/discourse/script/import_scripts/mbox/importer.rb:11:in `initialize'
7: from /var/www/discourse/script/import_scripts/mbox/support/settings.rb:8:in `load'
6: from /usr/local/lib/ruby/2.6.0/psych.rb:577:in `load_file'
5: from /usr/local/lib/ruby/2.6.0/psych.rb:577:in `open'
4: from /usr/local/lib/ruby/2.6.0/psych.rb:578:in `block in load_file'
3: from /usr/local/lib/ruby/2.6.0/psych.rb:277:in `load'
2: from /usr/local/lib/ruby/2.6.0/psych.rb:390:in `parse'
1: from /usr/local/lib/ruby/2.6.0/psych.rb:456:in `parse_stream'
/usr/local/lib/ruby/2.6.0/psych.rb:456:in `parse': (/shared/import/settings.yml): did not find expected key while parsing a block mapping at line 2 column 1 (Psych::SyntaxError)
Thanks @gerhard Sigh…I should have seen that issue, my first bout with Ruby. Now, I think I’m a little closer but a different error (see below). Since the import script is now loading Groups, etc., I assume the new error is past the initial problem. I also assume the referenced db file is import/index.db created by the import script (not created).
The mbox import is starting...
Loading existing groups...
Loading existing users...
Loading existing categories...
Loading existing posts...
Loading existing topics...
Traceback (most recent call last):
9: from script/import_scripts/mbox.rb:9:in `<main>'
8: from script/import_scripts/mbox.rb:10:in `<module:ImportScripts>'
7: from script/import_scripts/mbox.rb:12:in `<module:Mbox>'
6: from script/import_scripts/mbox.rb:12:in `new'
5: from /var/www/discourse/script/import_scripts/mbox/importer.rb:14:in `initialize'
4: from /var/www/discourse/script/import_scripts/mbox/importer.rb:14:in `new'
3: from /var/www/discourse/script/import_scripts/mbox/support/database.rb:10:in `initialize'
2: from /var/www/discourse/script/import_scripts/mbox/support/database.rb:10:in `new'
1: from /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:89:in `initialize'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:89:in `open_v2': unable to open database file (SQLite3::CantOpenException)
SYSTEM won’t allow me to edit my comment so I am submitting this reply instead.
EDIT: To close the loop…My Yahoo Group import is now working, at least to the point of indexing 9951 emails. I have not yet finished the full import so more to come. I have edited settings.yml many times and am now back to the original which suddenly seems to work! without the syntax error. I don’t understand why I have had numerous error messages that appear inconsistent to me. The original syntax error in the settings.yml is again a mystery. The above error mag makes no sense to me…sigh.
@gerhard. I think I have found a way easier method of doing exactly the same as your guide, but with no technical knowledge required nor need for admin access to any server. Let me know what you think.
We’ll be essentially configuring a mailinglist and then using an email archive to send past conversations in order. Those emails will be forwarded, but not like the “Forward” button on email clients (that would override the headers and mess up the indentation). What we want to do is to remail them (send as they had been sent to discourse in the first place).
Access to the previous email exchanges: someone who has stored it all on their email client and can volunteer to forward it – let’s call that person John Doe.
Time: the email email forwarding will be very slow so discourse can handle (perhaps a few days with a computer running uploading the emails – depending on the archive size)
Thunderbird client: We also assume here John Doe uses the email client “thunderbird”. It may be possible to do this with other clients but I haven’t looked.
The following guide uses two email addresses as placeholders. You need to replace them with your actual addresses.
[email protected] John Doe’s email (the person will forward the full mailing list archive)
[email protected] discourse email for forwarding emails to the category of the mailinglist (see setup 1. for how you get it)
Here’s a basic rundown of the instructions:
follow the guide on Create a read-only mailing list mirror to create a mirror of your mailing list
Note: this will only mirror your mailinglist going forward. You’ll still miss out on past conversations. That’s what the rest of this guide is for.
Change the way discourse forwards emails to (I’m not actually sure this is needed)
Edit the category’s settings and under the setting Custom incoming email address: add at the end of what’s there |[email protected].
The pipe here works like a , as to say that you also want [email protected] to be able to send to that category
John Doe installs on thunderbird the extension Mail Redirect.
This is because it’s no regular email forward. What this will do is send the email as if it had gone to the discourse’s email address in the first place instead of John Doe’s
John Doe goes to the extension’s settings and sets the following to 1 (default is 5)
This will make sure the replies arrive in order: otherwise discourse isn’t quick enough to realise that the replies are chained and just creates a new topic for every reply – but it will make the forwarding process very slow
John Doe selects all of the mailinglist’s past emails, right-clicks and clicks on Redirect. Then a new window will open and he add [email protected] as the Resend-to
After this John Doe’s email client will be slowly sending the email archives to discourse. Just check after some time to see if the discourse category is getting filled with some nostalgically old conversations.
Remove John Doe’s email from that category’s Custom incoming email address:setting (and don’t forget to remove the |)
Uninstall Mail Redirect extension – you’ll likely not need it again, or at the very least increase back the SMTP connections to 5.
We are trying to migrate our Mailman lists into an already running discourse instance. There are several private lists included for which we need permissions set for the corresponding category. When creating those categories before the import, all the posts for the private lists are added to “Uncategorized” (so automatically public).
So we have two alternative questions:
Is there a way to set permissions for the imported mailing lists (if they would be only admin-visible, it would already be sufficient for us) before import?
Is there a way to add the mailing list to an existing category (with preset permissions)?
gerhard:
Create a new directory in the import/data directory and restart the import script.
My discourse is the continuation of a Yahoo group, which itself was a continuation an AOL listserv. Last fall, in the face of the great Yahoo purge, I was able to download a .mbox archive of the Yahoo group, and import those messages following these instructions. I’ve now gotten a partial archive of the AOL listserv, and I’d like to import those messages as well.
Easy enough, right? Just make import/data/foo, put the messages there, and run the import script. But what I’m wondering about is if I later manage to get a complete (or a more-complete) archive. Can I just put those files into import/data/foo, run the import script again, and have it add the new messages to the same category?
Would it de-dupe? Or would I see multiple copies of messages that appeared in both archives?
Would it change the answer to this question if one, the other, or both of the archives lacked message-id headers?
Would a new import in the same category overwrite existing messages?
Most of my users are in mailing-list mode. If I don’t want to spam them with hundreds (or thousands) of notifications, not to mention run up an expensive Mailgun bill, I assume I’ll want to disable email site-wide while the import is going on?
miri64:
Is there a way to set permissions for the imported mailing lists (if they would be only admin-visible, it would already be sufficient for us) before import?
Unfortunately that’s not possible.
miri64:
Is there a way to add the mailing list to an existing category (with preset permissions)?
Yes, you can trick the import script into reusing existing categories.
./launcher enter app
rails c
# Use the category ID shown in the URL, for example
# it's 56 when the category's path looks like this: /c/howto/devs/56
category = Category.find(56)
# Use the directory name where the mbox files are stored. For example,
# when the files are stored in import/data/foo, you should use "foo" as directory name.
category.custom_fields["import_id"] = "directory_name"
category.save!
That’s unexpected. I’ve never seen that happen, but I’ve never tried to import into existing categories with permissions other than the default permissions.
If you can’t get it to work I’d suggest you post an announcement on your forum, switch your site into read-only mode, create a backup, restore the backup on a different server, run the import, configure the category permissions, create another backup and restore it on your production site.
danb35:
Can I just put those files into import/data/foo , run the import script again, and have it add the new messages to the same category?
Yes, you can. You might want to keep the import/data/index.db file around, just in case you want to have a look at the previously imported data, need to modify generated message IDs or whatnot…
danb35:
Would it de-dupe? Would it change the answer to this question if one, the other, or both of the archives lacked message-id headers?
Yes, it wouldn’t import already imported messages as long as the Message-ID header stays the same. You are out of luck if the Message-ID header is missing in only one of the archives. We use the MD5 hash of the message if the header is missing. You’d need to ensure that both messages either have the same Message-ID header or result in the same MD5 hash.
danb35:
Would a new import in the same category overwrite existing messages?
danb35:
Most of my users are in mailing-list mode. If I don’t want to spam them with hundreds (or thousands) of notifications, not to mention run up an expensive Mailgun bill, I assume I’ll want to disable email site-wide while the import is going on?
All outgoing emails are disabled during imports.
Google Groups
.
The import script most likely won’t work on systems with less than 4GB of RAM. Recommended are 8GB of RAM or more. You can scale back the RAM usage after the import if you like.
The following steps assume that you installed
Discourse on Ubuntu
and that you are connected to the machine via SSH or have direct access to the machine’s terminal.
Make sure you don’t have any pinned posts in your group, otherwise the crawler might fail to download some or all messages.
or 
