Creating a matrix from a long data.frame

There can never be too many examples for transforming data with R. So, here is another example of reshaping a data.frame into a matrix.

Here I have a data frame that shows incremental claim payments over time for different loss occurrence (origin) years.


The format of the data frame above is how this kind of data is usually stored in a data base. However, I would like to see the payments of the different origin years in rows of a matrix.

The first idea might be to use the reshape function, but that would return a data.frame. Yet, it is actually much easier with the matrix function itself. Most of the code below is about formatting the dimension names of the matrix. Note that I use the with function to save me a bit of typing.


An elegant alternative to matrix provides the acast function of the reshape2 package. It has a nice formula argument and allows me not only to specify the aggregation function, but also to add the margin totals.

Five things you should know about flood insurance

1) Your homeowners policy doesn't cover floods. Flood damage is not a covered peril on standard homeowners policies and most commercial policies, although many people assume that it is. That can be a costly assumption.

2) You can get an estimate of your property's flood risk online with a "one-step flood risk profile."

3) You may have to have flood coverage. Mortgage lenders often require flood coverage if a home is located in a flood-prone area (also known as a "special flood hazard area.")

4) Most people buy flood coverage through the government. Flood insurance is widely available through the National Flood Insurance Program, which is run by the Federal Emergency Management Agency, or FEMA. There are limits, however, on how much damage they'll cover. Many local agents sell NFIP policies.

5) Rates may be going up. In July 2012, Congress passed the Biggert-Waters Flood Insurance Reform Act, which will change the way the National Flood Insurance Program is run. Among those changes: premiums will increase for some policyholders. That's being done to make the program more financially stable.

Why Are You Afraid?

This is a blog. You, Dear Readers, are viewing this online either on the blog’s home site or on the AOL Patch System. You are comfortable online and this is probably only one of several blogs that you read on a regular basis. We know how to research issues. We know how to find the information we need to help us form our opinions or to confirm our long held beliefs.

Our comfort with the internet allows us to laugh at the television reports that paint an issue in strictly solid black or solid white. We can, if we wish, quickly go online and find the shades of gray.

But what if you don’t have internet access? What if you not only don’t have a computer, but you really don’t know how to use one? Then you are at the mercy of your news sources – TV, the newspaper, the radio, and your friends. And many of your news sources like to keep you nervous. Scared to death. Many, but not all.

Monica Robbins is the Senior Health Correspondent at WKYC, Channel 3. Ms. Robbins has been covering health issues in the Greater Cleveland market for over ten years. She and her station have worked diligently to demystify the Patient Protection and Affordable Care Act (PPACA) from its inception. The WKYC website and her on air reports have attempted to answer viewers’ questions in a straight-forward, non-political fashion.

In an effort to build on a mission of providing information instead of fear, Monica Robbins invited the local chapter of the National Association of Health Underwriters (NAHU) to spend three hours last night fielding viewers’ questions. Channel 3 set up a phone bank, brought in snacks and sandwiches, and promoted our availability online, during Dr. Phil, and on the evening news broadcast.

We received nearly 300 phone calls!

There was a common thread that ran through the vast majority of the calls that I received, FEAR. The people who called were afraid of “Obamacare”. How was the new law going to affect them? Would they lose their coverage? Will they be able to afford their new policies?

A third of my calls were from people on Medicare. Aside from the improvements in the Medicare Part D (Rx) benefits, Medicare is pretty much untouched by the PPACA. But my 11^th caller was a nervous 68 year old. His buddy told him that the premium for his Medicare Supplement was going to be four times higher next year due to Obamacare. Some friend. There was the 66 year old who was concerned about her vitamin D prescription. And there was the woman in her mid-fifties, still recovering from the surgeries to remove brain tumors, who had been told that the new policies would not cover liver transplants for anyone over 60. There is nothing in the law that prohibits liver transplants for 61 year olds, and she doesn’t need a transplant, but she has heard the reports and she is worried.

There wasn’t a single call, not one, from someone who will be hurt by the new law. My fifth call was a grandmother inquiring about coverage for her 22 year old granddaughter. The young woman is an uninsured college student. I had several calls from people who had lost their jobs, could not afford to exercise their COBRA option, and were now uninsured. Most had dependent children. The PPACA doesn't benefit everyone. What law could? But the concern and fear fueling these calls was a direct result of negative, often fact-less, political messaging.

Two of my calls came from people who were paying a lot of money for their or their spouse’s employer sponsored group health insurance plans. I couldn’t promise that the new health plans would be cheaper. I could reassure them that they will have choices and the opportunity to shop for alternate coverage.

Many of our callers just needed a safe website that could answer their questions. This one, created by the Ohio Association of Health Underwriters, even has a subsidy calculator. I patiently repeated the actual web address for them as they wrote it down before heading to the public library to access the internet.



We were supposed to start at 5 PM, but the phones started to ring before that. After an hour I turned to Ms. Robbins and Ingrid Martin, our Board Member who had helped to organize this, and asked when we were going to do this again. They had already agreed to the need. They appreciated our mutual commitment to helping Greater Clevelanders in making this transition. I’m sure we will be back again in a few weeks.

I, of course, am looking forward to doing this again. A veteran of numerous charity telethons, I love answering the phone on TV. That and we were too busy to meet Russ Mitchell when he came down to say hello. He and I share a real appreciation of Gino Vannelli.

DAVE

www.bcandb.com

We're looking for a communications and social media manager

Please help us spread the word - do you know a communications expert who's looking for a new challenge? We're currently recruiting for a Public Affairs Communications and Social Media Manager.

This position reports to the Deputy Commissioner for Public Affairs and manages select agency-wide public affairs strategies and communication projects. It also oversees the agency's social media efforts, represents the agency as senior writer and editor on legislatively required reports and high-profile projects for the commissioner and is primary spokesperson for news media and stakeholder groups on agency administrative, civil and criminal enforcement actions.

Here's the full job announcement. Please share with anyone you think might be interested.

We're taking applications through Oct.8.

How to contact Washington's Health Benefits Exchange

Earlier this month, the Washington Healthplanfinder (our state's health insurance exchange) opened its toll-free hotline to start answering questions about health coverage options, how to access financial help and what you need to know about the Exchange's enrollment process. The phone number is 1-855-923-4633 or TTY/TDD 1-855-627-9604. They're available from 7:30 a.m. to 8 p.m., Monday through Friday.

The Healthplanfinder can also help you find other people -- a broker in your local community, say, or a nearby in-person assister -- to help you through the process. Click on the link or image above to find out more.

Changing the width of bars and columns in googleVis

Changing the plotting width in bar-, column- and combo-charts of googleVis works identical and is defined by the bar.groupWidth argument. The dot in the argument means that it has to be split in R into bar="{groupWidth:'10%'}".

Example

library(googleVis)
cc <- gvisColumnChart(head(Population,10), 
                      xvar="Country", yvar="Population",
                      options=list(seriesType="bars", legend="top",
                                   bar="{groupWidth:'10%'}",
                                   width=500, height=450),
                      chartid="thincolumns")
plot(cc)

Session Info

R version 3.0.1 (2013-05-16)
Platform: x86_64-apple-darwin10.8.0 (64-bit)

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] googleVis_0.4.5

loaded via a namespace (and not attached):
[1] RJSONIO_1.0-3 tools_3.0.1 

A (Lack of) Progress Report

The car was careening down the hill. The driver, desperate to stop, never touched the brake pedal. Instead, he grabbed the emergency parking lever with both hands and violently pulled it towards himself. It snapped off! The car picked up speed. The driver opened the door and abandoned the car. He got up. Dusted himself off. And as he watched the out-of-control vehicle continue down the hill he declared that whatever happens, happens. It was not his problem since he was no longer driving the car.

The Patient Protection and Affordable Car Act (PPACA) was signed into law March 23, 2010. We have seen plenty of changes in how we pay for our health care delivery system over the last 3 ½ years. But the biggest changes are scheduled for October 1, 2013.

Are we ready?

If you are a regular reader of this blog, you know the answer – Not Quite.

Let’s start with the policies. As of right now, the cupboard is empty. Anthem, Medical Mutual of Ohio, etc… are waiting for final approval. Even if the policies are all approved on Monday, September 23^rd, how do they all get loaded into the various computer quoting systems in eight days? Brochures? Ads? Supporting documentation? Is it realistic to believe that everything can be up and running in a week?

The new online insurance shopping portal, the Exchange, is another area of concern. Once policies are loaded into the computers, how will we access them? The insurers and agents are still wading through a convoluted system that is being created on the fly. We are spending big bucks to create websites that will allow individuals to go to one site, plug in their information to determine their personal premium subsidies, and then apply for coverage. Those connections are not in place.

The government is also not ready. The software created to determine how much premium subsidy someone would receive is still not functioning at 100%. Brian Cook of the Centers for Medicare and Medicaid Services (CMS) states that they have been rigorously testing the software for a year. OK, but it is still failing. Many people will base their choice of coverage on the net price of the policies. If the subsidy calculations are wrong, we have the potential for real problems. Can this be resolved in the next week? That may be possible.

Those are some of the challenges we face to implement the President’s health care law. Are these insurmountable? Not really.

First and foremost, the October 1 date is not a drop-dead deadline. If everything was working, Americans could begin to purchase policies on October 1^st that would become effective on January 1, 2014. It doesn’t matter whether the policy is purchased on October 1^st or December 7^th, the policy is still not going to start until January. If the exchanges aren’t ready until November 1^st, we can still have people covered January 1^st.

The PPACA is a poorly written bill that attempted to do too much in too little time. I have talked to our elected representatives, staffers, and Health and Human Services (HHS) employees. They are not evil. They mean well, but they may be in over their heads.

And that brings us to Senator Ted Cruz. Mr. Cruz spends an inordinate amount of time talking about defunding “Obamacare”. Let’s be clear – we are not repealing nor defunding the PPACA. Mr. Cruz’s entire cynical campaign is a tactless fundraiser and a desperate cry for attention. Shockingly, even political commentators on FOX are now calling him out as a huckster.

The insurance industry has spent millions upon millions of dollars to comply with this law. The states have spent as much if not more. Every business, even ones with only one employee, has had to complete forms and make decisions about health insurance. Most of the policies for sale today will no longer be available in a few months.

Repeal the PPACA? Will all of the money, the time, and the effort of the last 3 ½ years be for naught? Which of the regulations stay on the books? Which disappear? Who will tell the sick that their insurance will cease on December 31^st?

The people leading the charge to repeal the PPACA or worse, to simply defund it, will only serve to make a muddled mess worse. They understand that the Patient Protection and Affordable Care Act could have been slowed or stopped by gently applying the brakes sometime over the last 3 years. But they never touched the brakes. Now the PPACA can’t be stopped.

The legislators who never tried to constructively modify the PPACA would like you to believe that they bear no responsibility for any of the problems. They jumped out of the car.

We’re at the bottom of the hill.

DAVE

www.bcandb.com

"I'm on Medicare Part A and B. I want to drop Part B and buy a health plan through the Exchange so that I can get a subsidy"

Don't do it.

That's worth saying again: Do Not Do This.

Here's why: Most health insurance plans have language in their policies that lets them drop anyone who is eligible for Medicare. As a result, even if you manage to sign up for the plan, the company will likely eventually figure out that you're eligible for Medicare and will drop you.

Then, if you go back onto Medicare Part B, you'll have to pay a penalty for as long as you continue to have Medicare. The penalty is 10 percent for each full 12-month period that you could have had Part B.

And that's not all. If you are Medicare-eligible and you purchase a plan offered on the Exchange, you are not eligible for an Exchange plan subsidy. (If you are on Medicare, you are already getting a subsidy, because the federal government pays far more in Medicare costs that current Medicare recipients paid into the program.)

"I just got a letter from my insurer saying that I have to switch health plans because of Obamacare. What can I do?"

<$BlogItemBody$>

"I was turned down for life insurance due to my health. Does this mean I can't get life insurance at all?"

<$BlogItemBody$>

Pierce County man charged with insurance fraud and attempted theft

A University Place man, Leandre Garner, has been charged by the attorney general's office with felony insurance fraud and second-degree attempted theft for filing a bogus claim with State Farm.

On Nov. 8, 2012, Garner got coverage online with the company for his 2007 Chrysler 300. Prior to that date, the vehicle was uninsured.

On Nov. 9, 2012, Garner said, he returned home from an appointment and discovered that his car had been hit by an unknown vehicle. The damage was estimated at $4,339. Garner filed a claim.

The problem: half a dozen people subsequently told investigators from State Farm and our Special Investigations Unit that Garner's car was damaged well before November. His ex-girlfriend said it happened around September, not November. A nearby tenant, an apartment office worker and an apartment groundskeeper also said the accident happened well before Nov. 8. So did the body shop that did the estimate.

Asked if he'd taken photos of the damage, Garner showed a State Farm investigator cell phone images he'd taken. He was surprised when the investigator pointed out that the metadata embedded in the images showed that they'd been taken Sept. 19, 2012.

When asked about this, Garner said that the must be a problem with the phone.

Arraignment is scheduled for Pierce County Superior Court on Sept. 24, 2013.

Using planel.groups in lattice

Last Tuesday I attended the LondonR user group meeting, where Rich and Andy from Mango argued about the better package for multivariate graphics with R: lattice vs. ggplot2.

As part of their talk they had a little competition in visualising London Underground performance data, see their slides. Both made heavy use of the respective panelling / faceting capabilities. Additionally Rich used the panel.groups argument of xyplot to fine control the content of each panel. Brilliant! I had never used this argument before. So, here is a silly example with the iris data set to remind myself of panel.groups in the future.

"I heard I can keep my adult child on my health insurance until age 26. But do I have to?"

Q: I heard I can keep my adult children on my health insurance until they turn 26. But what if I don't want to?

A: Then don't. Health care reform permits -- but doesn't require -- parents to keep their adult children on the parent's health plan up to age 26, unless the children have coverage through their own employer.

That said, you may want to provide coverage if you can afford it. No one is immune from bad luck, and rates for medical care when a person has no insurance can be very high indeed.

Also: if your child doesn't have a job or has a job that doesn't offer health coverage, you may be able to extend your coverage to him/her more cheaply than they could buy an individual policy on their own.

Zen And The Art Of Insurance Selling

<$BlogItemBody$>

Woman who pretended to be employer in lost-wage claim pleads guilty to insurance fraud

<$BlogItemBody$>

Common questions about Medicare and health care exchanges

We've been getting a lot of questions about whether the new health care exchanges affect people on Medicare.

The short answer is no. If you have health coverage through Medicare, you don't need to do anything. It will not affect your coverage.

Among the other questions we're hearing frequently:

Do I need to re-enroll in my Medicare plan through the new health insurance Exchange?

Nope. Medicare's open enrollment is not part of the Exchange. If you are on Medicare, do not sign up for a plan in the Exchange.

Will I lose my Medicare coverage due to health reform and the Exchange?

No. Health care reform and the Exchange do not affect your Medicare coverage. You still have the same benefits and security you have now with Medicare.

Will people on Medicare be fined for not buying a health insurance Exchange plan?

No. In fact, it's against the law for someone who knows you have Medicare to sell you an Exchange plan.

Can I go to the Exchange and get a subsidy to help pay for my Medicare coverage?

Sorry, but no. If you're on Medicare, you're not eligible for the subsidies, which are for people buying coverage through the Exchanges.

For more questions -- including how the Exchange and Medicare work for recent immigrants, for those about to turn 65, etc. -- please see our new "popular questions about Medicare and the Exchange" web page.

Changes mean quicker responses to your insurance complaints

We've launched a new complaint response system that's speeding up the time between consumer insurance complaints and resolutions.

As Washington state's insurance regulatory agency and an advocate for consumers, we help with thousands of consumer complaints each year. Typical complaints involve wrongly denied claims, delayed payments and cancelled coverage.

The new online system, which is a secure link between our office and insurance companies we regulate, allows us to quickly get those complaints (along with our questions or concerns) to insurers. They'll look into the case and often reconsider their initial decision.

For years, largely in the interest of protecting complainants' private information, this process was handled by mail. Insurance companies were allowed 30 calendar days to respond to a complaint. 

The new online system is also secure -- and it's dramatically faster. Now insurers must respond electronically within 15 business days.

In other words, we've cut the time to process consumer complaints against insurers by more than 25 percent.

Got a complaint about your insurer? You can file a complaint online or call our Consumer Hotline 1-800-562-6900.

ave and the "[" function in R

The ave function in R is one of those little helper function I feel I should be using more. Investigating its source code showed me another twist about R and the "[" function. But first let's look at ave.

The top of ave's help page reads:

Group Averages Over Level Combinations of Factors

Subsets of x[] are averaged, where each subset consist of those observations with the same factor levels.

As an example I look at revenue data by product and shop.

revenue <- c(30,20, 23, 17)
product <- factor(c("bread", "cake", "bread", "cake"))
shop <- gl(2,2, labels=c("shop_1", "shop_2"))

To answer the question "Which shop sells proportionally more bread?" I need to divide the revenue vector by the sum of revenue per shop, which can be calculated easily by ave:

(shop_revenue <- ave(revenue, shop, FUN=sum))
# [1] 50 50 40 40
(revenue_split_in_shop <- revenue/shop_revenue)
# [1] 0.600 0.400 0.575 0.425 # Shop 1 sells more bread than cake

In other words, ave has to split the revenue vector by shop and apply the sum function to it. Well that's exactly what it does. Here is the source code of ave:

#  Copyright (C) 1995-2012 The R Core Team
ave <- function (x, ..., FUN = mean)
{
    if(missing(...))
 x[] <- FUN(x)
    else {
 g <- interaction(...)
 split(x,g) <- lapply(split(x, g), FUN)
    }
    x
}

However, and this is what intrigued me, if I don't provide a grouping variable (missing(...)) it will apply the function FUN on x itself and write its output to x[]. That's actually what the help file to ave mentioned in its description. So what does it do? Here is an example again:

ave(revenue, FUN=sum)
# [1] 90 90 90 90

I get the sum of revenue repeated as many time as the vector has elements, not just once, as with sum(revenue). The trick is that the output of FUN(x) is written into x[], which of course is output of a function call itself "["(x).

I think it is the following sentence in the help file of "[" (see ?"["), which explains it: Subsetting (except by an empty index) will drop all attributes except names, dim and dimnames.

So there we are. I feel less inclined to use ave more, as it is just short for the usual split, lapply routine, but I learned something new about the subtleties of R.

Insurance questions: Does my homeowners policy cover lightning?

In most cases, yes. Lightning is a covered peril in standard homeowners policies. Typically, both direct physical damage -- like burns, shattered windows, melted wiring -- would be covered. And if the lightning sets your home on fire, your fire coverage would also kick in.

How about lightning-caused damage to your electronics, like a TV or computer? Also typically covered.

And what about your car? If that gets hit by lightning, is the damage covered? Again, in most cases yes -- IF you have comprehensive coverage. (A man riding his motorcycle near Chehalis yesterday was struck by lightning, but is apparently doing well, other than a partly melted helmet. Really.)

Health care reform questions: Where can I get help?

Q: Will there be health care advocates for people who are not able to understand the complexities of the health care process? I have a family member who cannot work and is in dire medical need and struggling with doctor and drug costs. What help will they get from the health care reform?

A: Yes, there definitely will be advocates to help people navigate the complexities of finding the right health coverage. Health care reform includes a network of navigators and other people to help, and many insurance agents and brokers can help as well. Here's a list of the organizations that have received grants to provide in-person assistance here in Washington state.

Here in Washington state, you can sign up with the Washington HealthPlanFinder to be contacted by assistance staff within the first two weeks of October. They can answer your questions and help with enrollment in the exchange, if that's the best option for you. The coverage would start in January 2014.

In this particular situation, with your relative struggling today to pay for medical care and prescription drugs, feel free to give our consumer advocacy staff a call. They can walk you through the options, including free or low-cost medical, vision and dental clinics, help paying for drugs, and how to appeal when a health insurer won't pay for a treatment or prescription.

The hotline number is 1-800-562-6900. (Don't live in Washington state? Here's how to find your own state's insurance regulator.) You can also email us at AskMike@oic.wa.gov.

WA: Two more Exchange health plans approved for King, Pierce, Spokane counties

From a press release we put out this morning:

FOR IMMEDIATE RELEASE – Sept. 4, 2013

Media contact: Public Affairs (360) 725-7055

Kreidler settles with another health insurer – approves two more Exchange plans for King, Pierce, Spokane counties

OLYMPIA, Wash. – Insurance Commissioner Mike Kreidler has reached a settlement with Molina Healthcare of Washington, Inc. (Molina) and approved its two plans for sale in Washington’s Health Benefit Exchange, the Washington Healthplanfinder.

Consumers in Washington will now have 43 choices in the Exchange when open enrollment begins Oct. 1. Molina’s two plans will be available in three counties: King, Pierce and Spokane.

Previously, Molina only participated in the Medicaid market. Its approval to sell inside the new Washington Healthplanfinder guarantees Medicaid enrollees continuity of care and creates even more competition in the marketplace.

Molina was one of five companies Kreidler disapproved for sale in Washington’s new Exchange. Molina, Coordinated Care Corp., Kaiser, and Community Health Plan of Washington (CHPW) all appealed Kreidler’s decision. Molina later dropped its appeal, but reactivated it Aug. 29.

The reactivated appeal allowed a settlement. Specifically, Molina corrected information in its provider contracts to gain approval.

Kreidler began discussions with only those companies he believed could make the necessary fixes in time before the federal deadline of Sept. 5. Ten plans from Kaiser and Community Health Plan of Washington were approved Aug. 30.

The Executive Board of the Health Benefit Exchange is schedule to certify the final list of approved plans today at 1 p.m. It is scheduled to submit its final list to the federal government Sept. 5.

“I made the tough decision to disapprove some plans on July 31 because I didn’t believe they were good for consumers,” said Kreidler. “I’m pleased that we’ve reached a settlement with some of these companies to bring more quality plans to the Exchange and that consumers will be protected.”

####

Doughnut chart in R with googleVis

The guys at Google continue to update and enhance the Chart Tools API. One new recent feature is a pie chart with a hole, or as some call them: donut charts.

Thankfully the new functionality is being achieved through new options for the existing pie chart, which means that those new features are available in R via googleVis as well, without the need of writing new code.

Doughnut chart example

With the German election coming up soon, here is the composition of the current parliament.

Session Info

R version 3.0.1 (2013-05-16)
Platform: x86_64-apple-darwin10.8.0 (64-bit)

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] googleVis_0.4.5

loaded via a namespace (and not attached):
[1] RJSONIO_1.0-3 tools_3.0.1