Last Post 06/07/2013 08:37 AM by Bandit292 (715) . 16 reply.

 (0) 
Sort Posts:

06/02/2013 07:47 PM

MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

I did my end of week data download today, and stumbled upon several GroupRS errors for Securities from several different Industry Groups. Just as an example, the Insurance-Diversified group has a current Rating of 12 which equates to a GroupRS of "A+" ... but every Security record downloaded from that Group has an "A" GroupRS value. Further down the Ratings, the Telecom-Infrastructure group has a current Rating of 17 which equates to a GroupRS of "A" ... but every Security record downloaded from that Group has an "A+" GroupRS value. These are just two examples of many Groups that have similar GroupRS data errors.

 

We pay good money for this tool called MarketSmith, and the value is in the data and screening capability. I have raised issues before about the data integrity, but have received NO feedback from anyone concerning those issues. MarketSmith needs to provide better data screening and validation before putting the information out for use. They also need to go public with any data issues detected so lookback data can be corrected by users.

 

Garbage in, garbage out. Is MarketSmith data accurate ... or is it littered with unvalidated errors???



06/02/2013 09:30 PM

Re: MarketSmith End of Week Group RS Data Errors
by Richard55 (65) Joined on 03/28/2013 Posts: 8

What do you mean you did a "data download"? Did you uncover this problem through the screening feature?



06/03/2013 09:55 AM

Re: MarketSmith End of Week Group RS Data Errors
by MARKETSMITH (12090) Joined on 06/14/2010 Posts: 952

 

One of our automatic data updates failed to run over the weekend, causing errors in a small set of data points. Our programmers are aware of this issue and have manually run these updates today. This should have corrected all data issues, but if you notice any further problems, please let us know and we will correct them immediately. We apologize for any inconvenience this may have caused.



06/03/2013 10:58 AM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

Richard55,

 

On Sat/Sun of every week, I run a series of screens and use the Export to Excel function of MarketSmith to collect end of week data. I use a small utility program to jam this data into an Access database for further use and analysis. That transfer process runs a series of simple data validation routines. Those validation routines kept choking on the error littered data from this weekend's downloads.

 

Anyone who ran a screen over the weekend using Group RS as a criteria, likely got faulty returns. Notice that the reply from MarketSmith (whoever that is) notes that they uncovered a problem ... but did not report it until I called them out on it. This is piss poor customer service for a data heavy service that does not come cheap. Notice also that they do not reveal exactly what errors they uncovered. My guess is that it is more extensive that they lead us to believe. If you did any screening prep over the weekend, you don't have any idea how faulty the results are because MarketSmith is not detailing what those errors are.



06/03/2013 11:09 AM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

MarketSmith (whoever that may be),

 

This is an unacceptable reply.

 

When the errors were detected, why was the discovery not communicated on the spot? Did you even discover the errors before they were presented to you? What, exactly, are those errors? How am I to rely on my weekend data pulls when I don't know where the errors are? How can I correct those errors for last week's data pulls ... you have no capability in the tool to pull up past data?

 

Everyone paying for MarketSmith needs some assurance that your data is valid and undergoes some detailed validation before it is posted for online consumption. You sell a data heavy service. Errors in the data are not acceptable. You need to get out front of any data errors with your customers ASAP and quit trying to cover up the problem and move on.

 

Additional errors discovered after my post last night include Group security count errors in, at least, these Groups: G3441 / G3442 / G6731. Where else can I expect to find data errors from my weekend data pulls?



06/03/2013 04:57 PM

Re: MarketSmith End of Week Group RS Data Errors
by MARKETSMITH (12090) Joined on 06/14/2010 Posts: 952

 

We pride ourselves on the quality and timeliness of our data, so we were surprised when this problem surfaced. As with any technology driven product, there may be occasional glitches like the one we experienced this past weekend. MarketSmith has a number of automatic quality checks and safeguards in place to track our data updates, unfortunately the error occurred in an area that failed to trigger an auto-alert to the team monitoring our service. The issue was with the update to our Relative Strength Rating, Industry Group Relative Strength Rating and U/D Volume Ratio. In addition to the message we received from you, the problem was identified by members of quality assurance team during their Monday morning audit. The issue was corrected immediately by our programming team and new alerts and additional safeguards have been put in place to prevent this from occurring in the future.


The MarketSmith customer support team typically notifies customers of technical issues during normal business hours, however in this case the issue was resolved and the data was updated by early Monday eliminating the need for a notification.


We apologize for this update error and the inconvenience it caused.



06/03/2013 06:12 PM

Re: MarketSmith End of Week Group RS Data Errors
by Richard55 (65) Joined on 03/28/2013 Posts: 8

Bandit -

Does the data you were looking at appear to be corrected and up to date?

Richard



06/04/2013 09:02 AM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

Thanks for the better reply concerning this issue, but ...

>> The MarketSmith customer support team typically notifies customers of technical issues during normal business hours, however in this case the issue was resolved and the data was updated by early Monday eliminating the need for a notification. <<

Do you really think this is an appropriate (non) action? You owe it to your paying customers to communicate any and all known data issues as soon as they are uncovered. Without this notification, members (like me) who used the screening tool during the timeframe when the data was littered with errors would have no idea their results were not valid. The decision not to notify customers looks and smells like an attempt to hide the issue from customers rather than facing up to it.

You have made a commitment to safeguard against data errors in the future. You also need to make a commitment to your customer base to communicate any data errors immediately upon detection with the scope of the findings and a follow-on communication when those issues are resolved.



06/04/2013 09:05 AM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

I did a data pull again last night before the Monday data update deadline. I will run the data through my validation routines later this evening and get back to you with any further issues.



06/04/2013 10:46 AM

Re: MarketSmith End of Week Group RS Data Errors
by Richard55 (65) Joined on 03/28/2013 Posts: 8

To the Customer Support Team:

Given that we have been instructed by the Market/Smith Coaches to do the bulk of our research on weekends, your explanation below is problematic. Can you assure us that this will not happen again & how can we be certain that the data is valid (what precautions are in place)?

>> The MarketSmith customer support team typically notifies customers of technical issues during normal business hours, however in this case the issue was resolved and the data was updated by early Monday eliminating the need for a notification. <<



06/05/2013 02:01 PM

Re: MarketSmith End of Week Group RS Data Errors
by MARKETSMITH (12090) Joined on 06/14/2010 Posts: 952

 

Our data update process is equipped with alerts that prompt our team, night or day, of any errors or malfunctions within the MarketSmith platform. The data error that occurred last weekend did not trigger an alert. We have discovered the cause and have corrected the problem.


MarketSmith has had minimal issues with data quality that required alert messages to our membership in the past, but we recognize that some members may want more information on a more timely basis. To that end, we have initiated a mechanism to allow our programming team to communicate directly to customers when technical issues arise outside normal business hours. Should such an issue occur, members will be notified using the homepage announcements area.


If you have any questions regarding our data updates, we invite you to call a MarketSmith Product Coach at (800) 424-9033.



06/05/2013 07:18 PM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

>> MarketSmith has had minimal issues with data quality that required alert messages to our membership in the past, but we recognize that some members may want more information on a more timely basis. To that end, we have initiated a mechanism to allow our programming team to communicate directly to customers when technical issues arise outside normal business hours. Should such an issue occur, members will be notified using the homepage announcements area. <<

Excellent. Thanks for taking definitive steps to ensure the integrity of the data we consume here.

That said, the membership should understand the scope of the data issues that were not communicated last weekend to appreciate the change you have announced. At least 21 data points (potentially more) that could be used as screening criteria or data output were effected, to include:

  - Price

  - Price % Change

  - Volume

  - Avg Volume

  - Vol % Change

  - EPS Rating

  - RS Rating

  - Group RS

  - SMR Rating

  - A/D Rating

  - Composite Rating

  - P/E

  - P/S

  - P/B

  - Yield

  - Alpha

  - Beta

  - ATR

  - Short Int %

  - Options

If you screened with or utilized any of the criteria listed above (potentially more) during the weekend, you did not have accurate data or results. Your MarketSmith customer support team also made the conscious decision NOT to communicate these errors to you when they were detected. The MarketSmith President has, to date, elected not to address these data errors, apologize for them, commit to correcting them or communicate any future errors in a timely manner. You pay for a data heavy service and flexible screening tool. Without accurate data, that tool is simply a toy.

The public acknowledgement of weekend data errors and a process to communicate any future errors by your customer support team is a huge step in the right direction. If management will follow up with a commitment to that effect, we can all feel more comfortable about the integrity of the data we consume via the MarketSmith tool.

As communicated to the MarketSmith team via private email, the end of week data they had supposedly fixed on Monday still contained 56 of 197 Group RS data errors and (at least) three Group security count errors. Bottom line - their data validation still needs to be refined. Even when they think the data is correct, it contains errors in prime screening criteria.



06/05/2013 07:50 PM

Re: MarketSmith End of Week Group RS Data Errors
by Richard55 (65) Joined on 03/28/2013 Posts: 8

Bandit,

Will you send me an email so we may discuss offline. 

Richard Feferman

tel: (858) 205-7424

email: richard@crarecovery.com



06/05/2013 07:58 PM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

On the way.



06/05/2013 08:26 PM

Re: MarketSmith End of Week Group RS Data Errors
by jjcalo (100) Joined on 03/01/2013 Posts: 5

Bandit292 - Thanks for this thread, your time and efforts and holding MarketSmith accountable.  Data integrity is a top priority.  Making money in stocks is hard enough!



06/06/2013 05:33 PM

Re: MarketSmith End of Week Group RS Data Errors
by MARKETSMITH (12090) Joined on 06/14/2010 Posts: 952

 

We have been working around the clock to identify and solve the issues discussed in this thread. Read more details on how it happened and how we will improve our responsiveness to members here.

 



06/07/2013 08:37 AM

Re: MarketSmith End of Week Group RS Data Errors
by Bandit292 (715) Joined on 08/11/2010 Posts: 127

Excellent!

Thanks for the positive change. Is there any way to indicate to the user when a data feed (update) process has started and when it has ended so the date associated with the data can be known to the user and data feed (update) transition periods can be avoided.

Thanks again.