Go Back    Forum > Featured > General Discussion

Reply
 
LinkBack Thread Tools Search this Thread
  #1  
08-01-2020, 07:20 AM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 14,389
Thanked 2,605 Times in 2,217 Posts
This site needs your help.

I need an "easy button" for some forum admin tasks. The manual methods are just too unwieldy.


The bounty:

I'm not necessarily asking for free help here -- though it would be nice. Perhaps you can return the favor for my past advice that helped you?

If not, I can offer:
- free Premium Membership (with benefits growing after new site is launched, price increasing)
- discount on my hardware
- some money
- a favor, to be used at a later date, and that's never a bad thing to have with me
- some tape conversion work, either discounted or free (least preferred, but will do it begrudgingly)
- some combination of the above

So, with that out of the way, onto the help needed...


The reason:


This is mostly so we can start sending out regular newsletters again. There is important info that needs to be communicated, be it warnings on Win10 updates mass killing capture cards, new drivers, new developments in video, etc. Important stuff. Not too often, brief, useful.

Also zap spam.


The 1st problem:

We can export member profile emails from vBulletin. But it's just raw data.
Code:
john@gmail jack@hotmail sally@gmail etc
It can be dumped in Notepad++, and space replaced with return. Then dump into Excel and sorted, but that's not always how I need it sorted.

We often need to sort by domain, all data after the @ symbol.
That can be done "easy enough" (manual work, time consuming) with a mix of yet more Notepad++ replacements and Excel. Then after sort, I have to again used Notepad to turn back into whole addresses.

I really need to cut down my admin workload.


The 2nd problem:

Forums have a problem with both spam and "disposable" emails. We do a really good job of keeping those worthless a-holes out, but some still manage to slip through (though usually not for long).

The easiest way to detect those is by looking at the domains and TLDs (.com, .xyz, etc). So that means splitting an email after the @, and also before the .com (but NOT all periods, as ".co.uk" is a single TLD, and that always messes up the Notepad++/Excel sorting)

The subdomains also always mess up sorting (example= @tx.rr.com, or spammer @ahole.163.in). So in programming terms, I'm assuming both compound domains and TLDs need from-end-of-domain sorting instructions, not from-start. In the above example, all @_.rr.com and @_.163.in are the sorted TLDs.

I'd also like to be able to show only domains as a single entry, in addition to the full list. For example, all @gmail can be shown in a sorted list. Or only a single @gmail entry is shown, because all duplicates are ignored, so I can see a simple list of just the various domains being used here. It's too easy to overlook smaller domains when you have elephants in the room, so remove or condense those elephants.

When seen, I can just manually delete the member in vBulletin.

And yes, we know lots of spammers use Gmail. But that eventually sorts itself, usually via bouncing or lack of activation.


The 3rd (and final) problem:

As you may know, as I try to often point out, this site attracts many professional and business users, not just home/hobby users. And those pros and businesses often do not use freemails (gmail, hotmail, etc). The ISP emails can go either way (@comcast, etc). They use there company/business emails in most cases.

I need to be able to extract and communicate only with these members. Some sort of option that automatically removes all @gmail, @hotmail, @gmail, @aol, @yahoo, etc, in a "pro user only" mode. And then also lets me add manual exclusions (exclusion list, hopefully in-script and not via external text files) if I decide that none of the @isp emails need to included this time.

Then I can appropriately only send emails to @hollywoodstudio, @b2b, @conversionbusiness, etc.


The script wanted:

These days, everything is "online script", so I'm just assuming this will be written in PHP or something. And we have development space, if needed.

I need a versatile script. (Or maybe several scripts?)
I need to be able to dump in all emails.
Then sort based on criteria needed. Sometimes I'm just looking for spammers or disposables to zap. Other times, I want to send an email only to pro/business members (using non-freemails, maybe non-ISP).

If I notice TLDs not mentioned here, like @protonmail, @gmx, etc), I can manually add those into the script (array?) myself. I can edit scripts, but not write them.

The output is the whole emails again.


The unlikely bonus:

This is less likely, probably more of a dream. But if this could be integrated into vBulletin, that would be awesome. We can setup a test environment. I'd also up the bounty.

Maybe not all features could be built into vBulletin, but just some, mostly the ability to exclude @freemail users from some Free Member newsletters. Even that single vB task would be a win, and may simply be an edit of the email members template; perhaps an added call, array, something?

But I would be happy with non-integrated script/solution. The vB stuff is just icing.


Questions?

Post questions here, any clarifications needed. Then we can move to PM/email as needed.

I really hope somebody(ies) here has the skills to do any/some/all of this. This is too far outside my skills.

Note: We will NOT be giving out actual email data. That goes against our policies. You will need to create dummy data. I can, however, provide dummy data suggestions, based on our actual email lists.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
08-16-2020, 12:04 AM
nicholasserra nicholasserra is offline
Free Member
 
Join Date: Aug 2017
Posts: 126
Thanked 12 Times in 11 Posts
I can probably knock this out quickly, if a little python CLI script is okay with you. Give me an example of the raw input i'll be working with.
Reply With Quote
  #3  
10-30-2023, 07:52 AM
antonio.casoria antonio.casoria is offline
Free Member
 
Join Date: Oct 2023
Posts: 2
Thanked 0 Times in 0 Posts
You can use regular expressions to filter out spam and disposable emails. For example, you can filter out common disposable email domains and TLDs (Top-Level Domains). Here's a simplified example in Python:
import re

emails = ["john@gmail.com", "jack@hotmail.com", "sally@gmail.com", "spammer@disposable.com"]

for email in emails:
if not re.search(r'@(gmail\.com|hotmail\.com)$', email):
print(email) # This email is not from Gmail or Hotmail


To send emails only to professional/business users, you can create a filter based on email domains for CRM data enrichment. Here's a simplified example:
professional_domains = ["@hollywoodstudio", "@b2b", "@conversionbusiness"]
exclusions = ["@gmail.com", "@hotmail.com", "@aol.com", "@yahoo.com"]

for email in emails:
domain = email.split('@')[1]
if domain in professional_domains and domain not in exclusions:
print(email) # Send email to this professional user

Last edited by antonio.casoria; 10-30-2023 at 08:09 AM.
Reply With Quote
  #4  
09-26-2024, 05:42 AM
charlotte.king
Guest
 
Posts: n/a
If you want to take it further, we could explore integrating this functionality into vBulletin, though that would require modifying the email templates and adding calls to your custom script for member filtering.
Reply With Quote
  #5  
09-26-2024, 09:00 AM
HomeVideoProject HomeVideoProject is offline
Premium Member
 
Join Date: Sep 2023
Location: Boston, MA
Posts: 16
Thanked 0 Times in 0 Posts
Do you have access to phpMyAdmin for this forum? If so, I can prepare a few SQL scripts that will take care of the above scenarios. I'll defer to somebody else to take the SQL and turn it into a vBulletin-authenticated page for easier access.
Reply With Quote
  #6  
10-10-2024, 09:37 PM
thecoalman thecoalman is online now
Premium Member
 
Join Date: Jan 2005
Location: United States
Posts: 143
Thanked 19 Times in 17 Posts
My full post is blocked as well as attached text file because of code, it's in the attached zip file. It has SQL for what you asked. There is also sample php script for outputting some HTML based on whatever the SQL is.


Quote:
Originally Posted by lordsmurf View Post
This site needs your help.

I need an "easy button" for some forum admin tasks. The manual methods are just too unwieldy.
This version of Vbulletin doesn't have wild card matching when searching for users email? phpbb does

Quote:
It can be dumped in Notepad++, and space replaced with return. Then dump into Excel and sorted, but that's not always how I need it sorted.
I don't use Excel, can't you just import the SQL? There is option for exporting Excel CSV in phpMyadmin and many others.

For notepad++ if you only need single column with each value on new line on the export page select CSV, then select custom and down the bottom remove the , and " for the three column options.



....,post continues in .zip file


Quote:
integrated into vBulletin,
I can do practically anything possible with phpBB. Most of things you tying to do here can be done in phpBB's ACP and if not directly it can be done with some SQL results. e.g. for your professional group you'd just have to output the username on single line using CSV as described above. Go into ACP, add a "professional" group and paste the entire list of usernames into add user panel. Then you can mass email the group.


Now you have incentive to sell me the top metal cover on 9911 off of one of those parts machines you have.


Attached Files
File Type: zip smurf.zip (1.8 KB, 1 downloads)

Last edited by thecoalman; 10-10-2024 at 10:04 PM.
Reply With Quote
  #7  
10-11-2024, 11:27 AM
thecoalman thecoalman is online now
Premium Member
 
Join Date: Jan 2005
Location: United States
Posts: 143
Thanked 19 Times in 17 Posts
FYI, if you interested in using that script I can make it more user friendly fairly easy. Multiple selections for what SQL is executed and presented in nice table format. Knowing the structure of the user table would be helpful, export it using phpMyadmin. Structure only doesn't contain any user data.

I don't need any payment as it's trivial job for me. That cover would be nice though if you have extra one laying around on parts machine. The circuit board I need would also be nice. I'll pay for them, discounts accepted.
Reply With Quote
  #8  
10-11-2024, 11:38 AM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 14,389
Thanked 2,605 Times in 2,217 Posts
Quote:
Originally Posted by thecoalman View Post
FYI, if you interested in using that script I can make it more user friendly fairly easy. Multiple selections for what SQL is executed and presented in nice table format. Knowing the structure of the user table would be helpful, export it using phpMyadmin. Structure only doesn't contain any user data.
I'll look this post over soon.

Quote:
I don't need any payment as it's trivial job for me. That cover would be nice though if you have extra one laying around on parts machine. The circuit board I need would also be nice. I'll pay for them, discounts accepted.
Remind me what you're referring to here.

PM if needed.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
  #9  
10-11-2024, 01:00 PM
thecoalman thecoalman is online now
Premium Member
 
Join Date: Jan 2005
Location: United States
Posts: 143
Thanked 19 Times in 17 Posts
I need the top case cover for 9911, mines all bent. I also need the circuit on the stator. Obviously something you'd only give up if it's on parts machine. There is apparently multiple models with same board. I have one but it's nearly "new" working machine of the lower priced variety.

https://www.digitalfaq.com/forum/mar...html#post93979
Reply With Quote
  #10  
10-16-2024, 11:01 PM
Winsordawson Winsordawson is offline
Free Member
 
Join Date: Sep 2010
Location: Behind you
Posts: 474
Thanked 30 Times in 26 Posts
It seems like you have a lot of answers here. I'll just add that for the first problem, it's rather simple in Excel. Just use Find and Replace the space with a new line, then use TEXTAFTER to get only the email domains. Or with RIGHT(A1, LEN(A1) - FIND("@", A1)), if your version does not support TEXTAFTER.

Unless you are looking for something else.
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Sorting and cataloging digital pictures, automatic, face detection? Reddwarf4ever Photo Processing, Scanning & Printing 10 02-05-2018 03:17 AM
How to create a MySQL database in cPanel? Brent Web Hosting 0 08-25-2013 08:46 PM
Websimon Tables plugin customized with table sorting [DOWNLOAD] kpmedia Website and Server Troubleshooting 0 11-06-2012 06:58 PM
Google Apps email vs cPanel server email? Which is best? prado Website and Server Troubleshooting 1 06-27-2012 04:31 AM

Thread Tools Search this Thread
Search this Thread:

Advanced Search



 
All times are GMT -5. The time now is 08:00 AM